nystudio107
Add support for llms.txt (similar to robots.txt, humans.txt, ads.txt, security.txt support)
Is your feature request related to a problem? Please describe.
We want an easy way to add llms.txt to our sites
Describe the solution you would like
Add a section in Global SEO settings for populating an llms.txt file
This has been brought up before. I'm not against adding it, but llms.txt is not a widely supported standard:
As of mid-2025, major AI companies like OpenAI, Google, Microsoft, and Meta do not officially or natively support the llms.txt standard in their primary AI products. The file is a proposed convention, not a widely adopted industry standard.
https://ppc.land/llms-txt-adoption-stalls-as-major-ai-platforms-ignore-proposed-standard/
So I've had some time to reflect on this.
I'm very conflicted, in that I don't want to be seen as endorsing something that is essentially a placebo at this point.
People need to understand that no major AI vendor actually supports or respects anything in llms.txt, and my implementing it in SEOmatic, I feel, may mislead people into thinking otherwise.
That said, people seem to want it anyway.
So I think what I will do is implement it, but put an explicit warning on the page that no major AI vendors support it currently, and that they need to implement bot control in robots.txt in order for it to be at least nominally effective.
Related: https://www.searchenginejournal.com/google-says-llms-txt-comparable-to-keywords-meta-tag/544804/
So I'm actually starting to think these may be better ideas/more likely to be adopted:
https://rslstandard.org/
https://datatracker.ietf.org/wg/aipref/about/
Isn't llms.txt just meant to be a machine-friendly version of a page? At least that's what I found out here: https://llmstxt.org/
Edit: somebody already made a plugin that interprets llms.txt this way: https://plugins.craftcms.com/llmify?craft5
Isn't llms.txt just meant to be a machine-friendly version of a page? At least that's what I found out here: https://llmstxt.org/
Edit: somebody already made a plugin that interprets llms.txt this way: https://plugins.craftcms.com/llmify?craft5
Yes, but still:
Right now, no major LLM provider officially supports llms.txt. Tools like GPTBot (OpenAI), Claude (Anthropic), and Google’s AI crawlers don’t reference or follow it as part of their crawling behavior. Some companies like Anthropic publish llms.txt files themselves, but there’s no evidence that any crawler is actively using them in retrieval or training.
ref: https://yoast.com/what-ai-gets-wrong-about-your-site-llms-txt/
So in other words, it's largely a pointless exercise at the moment.