You may have heard that every website now needs an llms.txt file for AI. The file is quick to make, but it does less than many people expect. This guide explains what it is in plain words, shows an example for an aesthetics clinic in Lahore, and says when it is worth adding.
What is llms.txt?
llms.txt is a plain text file, written for AI tools, that lists a website's most useful pages with a short note on each.
The name explains the idea. LLM stands for large language model, the kind of AI behind tools such as ChatGPT and Gemini. So llms.txt is a text file for LLMs.
The file sits at the root of a website, the top level of its address, so anyone can open it at yourwebsite.com/llms.txt. It is written in Markdown, a simple way to format text: a line that starts with # is a heading, and a line that starts with - is an item in a list.
Jeremy Howard proposed the format on 3 September 2024. It is published at llmstxt.org (opens in a new tab), and it is a proposal, not an official web standard.
Why was llms.txt proposed?
Because web pages are hard for AI tools to read cleanly. A normal web page wraps its words in menus, adverts and code. The proposal says turning that back into clean text is "difficult and imprecise".
AI tools can also read only a limited amount of text at a time, and the proposal says that limit is still too small for most whole websites. llms.txt is meant as a shortcut: one short file with the main facts and the most useful links.
What goes in an llms.txt file?
An llms.txt file has up to five parts, in this order:
- The name. A line that starts with #, such as "# Example Aesthetics Clinic". It is the only part that is required.
- A short summary. A line that starts with >, saying what you do and where.
- Optional notes. A few lines of plain text with more detail.
- Lists of links. Each list has its own heading starting with ##. Each link can have a short note after a colon.
- An "Optional" list. Links an AI tool can skip when it needs less detail.
Here is an example for a made-up aesthetics clinic in Lahore:
# Example Aesthetics Clinic> Example Aesthetics Clinic is a skin and laser clinic in Gulberg, Lahore. Its doctors offer laser hair removal, facials and acne-scar treatment. Appointments by phone or WhatsApp, Monday to Saturday. ## Treatments- [Laser hair removal](https://example.com/laser-hair-removal): sessions, prices and aftercare- [Acne-scar treatment](https://example.com/acne-scars): who it suits and how many sessions it takes- [Facials](https://example.com/facials): types, prices and how long each one takes ## Clinic- [Our doctors](https://example.com/doctors): names and qualifications- [Prices](https://example.com/prices): every treatment, in Pakistani rupees- [Contact and location](https://example.com/contact): address, map, phone and WhatsApp ## Optional- [Skin care articles](https://example.com/articles)
Three choices in it are worth copying:
- The summary states facts. It says what the clinic does, where it is and how to book, in two plain sentences.
- Every link goes to a real page. An AI tool that follows a link finds the full answer there.
- The doctors are listed. A page with the doctors' names and qualifications lets anyone check who gives the treatments.
llms.txt vs robots.txt vs a sitemap: what is the difference?
The three files do different jobs. robots.txt sets rules for crawlers, a sitemap lists your pages for search engines, and llms.txt is a short reading list for AI tools.
| Item | robots.txt | Sitemap | llms.txt |
|---|---|---|---|
| What it is | robots.txtRules for crawlers | SitemapA list of your pages | llms.txtA short reading list for AI tools |
| Who reads it | robots.txtSearch engine and AI crawlers | SitemapSearch engines | llms.txtSome AI tools, if they look for it |
| What it does | robots.txtTells crawlers which pages they may visit | SitemapTells search engines which pages exist | llms.txtPoints AI tools to your most useful pages |
| Does Google Search use it? | robots.txtYes | SitemapYes | llms.txtNo, Google Search ignores it |
A crawler is a program that visits web pages and reads them. Google describes robots.txt as a file that "tells search engine crawlers which URLs the crawler can access on your site." It describes a sitemap as "a file where you provide information about the pages, videos, and other files on your site."
What doesn't llms.txt do?
- It doesn't help you rank in Google. Google's guide (opens in a new tab) says Google Search ignores llms.txt, so the file will "neither harm nor help" your visibility there.
- It doesn't control crawlers. That is the job of robots.txt. llms.txt can't block or allow any bot.
- It doesn't make AI tools read it. Google's John Mueller has compared it to the keywords meta tag (opens in a new tab), an old hidden list of keywords that search engines learned to ignore. He also noted that AI services didn't use it.
- It doesn't replace good pages. AI tools still read your pages. If a page is thin or out of date, listing it doesn't help.
Key points
- llms.txt is a reading list for AI tools, not a way to rank higher.
- robots.txt, not llms.txt, decides which bots can visit your website.
- Clear, accurate pages do more than any summary file.
Why do some websites add one anyway?
Because it takes little time, and some tools now check for it. In May 2026, version 13.3 of Lighthouse, Google's free tool for testing web pages, added an llms.txt check to a new Agentic Browsing section. That section is about helping AI agents, tools that browse websites on a person's behalf, find their way around a site. It is separate from Lighthouse's SEO checks (opens in a new tab), and it has nothing to do with rankings.
Google's guide also says it is "completely fine" to keep an llms.txt file for other services that use one. We keep one on our own website, treat it as optional, and spend our time on the pages it points to.
How do you make an llms.txt file?
- Pick your 10 to 20 most useful pages. For a clinic, that means treatments, prices, doctors and contact details.
- Write the name and a short summary. Use facts: what you do, where, and for whom.
- Group the links under ## headings, such as Treatments and Clinic.
- Add a short note after each link. One line on what the page answers.
- Save the file as llms.txt at the root of your website. It should open at yourwebsite.com/llms.txt as plain text.
- Check that it opens without errors. Lighthouse flags an error if the file fails to load.
- Update it when your pages change. An out-of-date file sends AI tools to the wrong pages.
The words in this guide, explained
- LLM. Short for large language model, the kind of AI behind tools such as ChatGPT and Gemini.
- Text file. A file that holds only plain text, with no design or images, so any program can read it.
- Markdown. A simple way to format plain text: # starts a heading, - starts a list item and words (opens in a new tab) makes a link.
- Root. The top level of a website's address, such as yourwebsite.com. A file at the root opens at yourwebsite.com/filename.
- Crawler. A program that visits web pages and reads them for a search engine or an AI tool.
- robots.txt. A file at the root of a website that tells crawlers which pages they may visit.
- Sitemap. A file that lists a website's pages, so search engines can find them.
- Lighthouse. A free Google tool that tests web pages for speed, accessibility, SEO and more.
- AI agent. An AI tool that can browse websites and complete tasks for a person.
- Keywords meta tag. An old, hidden list of keywords in a web page's code. Search engines stopped using it years ago.
llms.txt: questions people ask
Not for Google Search. Google says Google Search ignores llms.txt, so the file will neither help nor harm your rankings there. Rankings come from pages that Google can crawl, index and trust.
No. robots.txt tells crawlers which pages they may visit. llms.txt is a reading list for AI tools, and it can't block or allow anything. To control AI crawlers, use robots.txt.
At the root of your website, so it opens at yourwebsite.com/llms.txt. The proposal also allows a file inside a folder, such as yourwebsite.com/docs/llms.txt, for the pages in that folder.
No. It is optional. A small website gets more from clear, accurate pages. If you want one anyway, it is quick to make.
It can, but it comes last. First make sure your treatment, price and doctor pages are accurate, up to date and checked by your doctors. Then a short llms.txt can point AI tools to them. See how we work with clinics and wellness practices.
What you do, where you do it and who for, in one or two plain sentences. Use facts a reader can check, such as your city, your services or your opening days, not slogans.
Sources
- llmstxt.org: the /llms.txt proposal (opens in a new tab): what the file is, who proposed it, why, and its format
- Google Search Central: optimising your website for generative AI features (opens in a new tab): Google Search ignores llms.txt, and keeping one is "completely fine"
- Google Search Central: robots.txt introduction (opens in a new tab): what robots.txt does
- Google Search Central: learn about sitemaps (opens in a new tab): what a sitemap is
- Search Engine Journal: Google's llms.txt guidance depends on which product you ask (opens in a new tab): John Mueller's comparison, and the Lighthouse 13.3 check (May 2026)

