← all agents

CCBot

Open crawl dataset that feeds many model training runs.

Operator Common CrawlTraining crawlerrobots.txt policy: reported to respectverifiable via published IP rangesvendor documentation ↗

Matches user-agent tokens: CCBot· source: curated from vendor docs

This directory documents crawler identity and declared purpose. It does not infer usage, model activity or web-wide volume from visits to gcdTracker.

Explore published crawler traffic →