AI readiness at your root
robots.txt + llms.txt, 26 crawlers checked
Two files decide what an AI engine can do with your site: robots.txt says who may come in, llms.txt says what is worth reading. We split the crawlers into the ones that power answer engines — and can cite you — and the ones that only take training data. Most tools lump them together and call any block a problem. It isn't.
Two requests: /robots.txt and /llms.txt