ApifyWebsiteContentCrawler
ApifyWebsiteContentCrawler is a web crawler by Apify that extracts and downloads full website content for use in AI, data analysis, and automation workflows. More info can be found at https://knownagents.com/agents/apifywebsitecontentcrawler
Operator Unknown operatorAI tooling / data providerrobots.txt policy: unknownno published IP rangesvendor documentation ↗
Matches user-agent tokens: ApifyWebsiteContentCrawler· source: ai.robots.txt list
This directory documents crawler identity and declared purpose. It does not infer usage, model activity or web-wide volume from visits to gcdTracker.