AIWebIndex
Builds an index of public pages and serves them to AI agents as extracted, readable text with attribution and a link back to the source. Does not train foundation models on crawled content. Identity can be checked three ways: published IP ranges at https://lyrenth.com/bot/ip-ranges.json, forward-confirmed reverse DNS under lyrenth.com, and Web Bot Auth signatures (RFC 9421). Full policy at https://lyrenth.com/crawler-policy
Operator [Lyrenth](https://lyrenth.com)AI search indexrobots.txt policy: reported to respectno published IP rangesvendor documentation ↗
Matches user-agent tokens: AIWebIndex· source: ai.robots.txt list
This directory documents crawler identity and declared purpose. It does not infer usage, model activity or web-wide volume from visits to gcdTracker.