llms.txt
llms.txt is a proposed convention for a Markdown file that summarizes a website so that large language models can understand its content easily.
llms.txt is a Markdown-formatted text file placed at the root path of a website that gives large language models (LLMs) and AI agents an overview of the site along with links to its key documents, so they can grasp its main content easily. It was proposed in 2024 by Jeremy Howard, a co-founder of fast.ai.
Unlike robots.txt, which tells search engine crawlers what they may access, or sitemap.xml, which lists a site's pages, llms.txt aims to provide concise, human-readable information that a language model can consult directly at inference time. It is not a standard set by a formal standards body, and it is used on sites such as the technical documentation of developer tools.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.