LLM friendly docs: llms.txt
The llms.txt file defines structured content and navigation guidance specifically for AI models. It's a proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time. Different purpose-specific files like robots.txt and sitemap.xml, llms.txt optimizes content comprehension and interaction for language models.
This PR implements a custom plugin for Docusaurus to generate two files into the build folder: llms.txt and llms-full.txt (and these files will be accessible through docs.lukso.tech/llms.txt & docs.lukso.tech/llms-full.txt)
Content details of these files:
- /llms.txt: Provides a high-level, structured overview of documentation to help LLMs quickly grasp the site’s organization and navigation paths.
- /llms-full.txt: Offers a complete, consolidated version of all documentation content to support in-depth processing by AI models.
To check out the content of llms & llms-full files, simply build the project with yarn build and check the content of the generated /build folder. Or use yarn serve after the build and visit localhost:3000/llms.txt
Upon the approval of this PR, I will submit our docs to be added to https://llmstxthub.com/