docs icon indicating copy to clipboard operation
docs copied to clipboard

LLM friendly docs: llms.txt

Open mustafademiray opened this issue 8 months ago • 1 comments

The llms.txt file defines structured content and navigation guidance specifically for AI models. It's a proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time. Different purpose-specific files like robots.txt and sitemap.xml, llms.txt optimizes content comprehension and interaction for language models.

This PR implements a custom plugin for Docusaurus to generate two files into the build folder: llms.txt and llms-full.txt (and these files will be accessible through docs.lukso.tech/llms.txt & docs.lukso.tech/llms-full.txt)

Content details of these files:

  • /llms.txt: Provides a high-level, structured overview of documentation to help LLMs quickly grasp the site’s organization and navigation paths.
  • /llms-full.txt: Offers a complete, consolidated version of all documentation content to support in-depth processing by AI models.

To check out the content of llms & llms-full files, simply build the project with yarn build and check the content of the generated /build folder. Or use yarn serve after the build and visit localhost:3000/llms.txt

Upon the approval of this PR, I will submit our docs to be added to https://llmstxthub.com/

mustafademiray avatar Apr 07 '25 15:04 mustafademiray

Deployed with Cloudflare Pages :cloud: :rocket: :ok:

github-actions[bot] avatar Apr 07 '25 15:04 github-actions[bot]