TIL: like robots.txt in a website, but for LLM

Just as a ‘robots.txt’ file directs web scrapers, a llm.txt file is there to direct our new AI overlords.

The famous ‘lighthouse’ accessibility scoring now includes checks for it too - a new category called ‘Agentic Browsing’, which checks how well AI agents can read and operate the site. On Chrome 150+:

DevTools > Lighthouse > tick Agentic Browsing > Analyze page load

The new standard: https://llmstxt.org

and here’s a tool for making one: https://design.dev/ai/llms-txt-generator/