vitepress icon indicating copy to clipboard operation
vitepress copied to clipboard

Support llms.txt generation

Open S1M0N38 opened this issue 8 months ago • 6 comments
trafficstars

Is your feature request related to a problem? Please describe.

VitePress lacks support for the emerging llms.txt "standard", which provides markdown content in a format optimized for LLM consumption. Documentation sites need a way to make their content more easily digestible by AI assistants for improved accuracy and context preservation.

Describe the solution you'd like

Implement native llms.txt generation in VitePress:

  • Auto-generate LLM-optimized versions of documentation
  • Support structured content formatting for better AI comprehension
  • Provide configuration options to customize output format

Describe alternatives you've considered

Manual content transformation or post-processing scripts, but these create maintenance overhead and version inconsistencies.

Additional context

The llms.txt standard focuses on making documentation more accessible to AI systems through optimized formatting and structure. Supporting this would enhance how VitePress documentation is consumed by AI assistants, resulting in more accurate responses when users query documentation content through LLMs.

Validations

S1M0N38 avatar Mar 02 '25 17:03 S1M0N38

Hello 👋

I created vitepress-plugin-llms which generates LLM-Friendly documentation

This plugin generates the following files:

.vitepress/dist
├── ...
├── llms-full.txt            // A file where all the website documentation is compiled into one file
├── llms.txt                 // The main file for LLMs with all links to all sections of the documentation for LLMs
├── markdown-examples.html   // A human-friendly version of `markdown-examples` section in HTML format
└── markdown-examples.md     // A LLM-friendly version of `markdown-examples` section in Markdown format

This plugin is still very new and under active development, not yet published, I will release the first release when I add tests (± tomorrow)

You can star the project, I would be very grateful for your support!

okineadev avatar Mar 16 '25 06:03 okineadev

The current solution for this is vitepress-plugin-llms (https://github.com/vuejs/vitepress/issues/4545#issuecomment-2796036135)

The plugin is now quite stable and is already used by many popular websites.


Regarding embedding llms.txt generation into VitePress itself - I am against it because:

  1. According to the UNIX philosophy, "A program should do one thing, and do it well." ^1
  2. Make modular and extensible programs ^2

Moreover, this will mean wasted time integrating functionality + many more dependencies for VitePress

I suggest the following option: transfer vitepress-plugin-llms to the @vuejs organization so that I (as the author of the extension) and the members of the organization can maintain the plugin, and that would be good!

okineadev avatar Apr 11 '25 09:04 okineadev

It ultimately depends on how many users need that feature. Enabling llms: true in vitepress config is significantly more convenient for end-users compared to installing and setting up a separate plugin. The same applies to other popular vitepress features like mathjax/katex support, local search, sitemap generation - and we will likely bring back auto-sidebar mode as well. While all of these could be (and were/are) achieved with third-party plugins, the built-in options offer a much smoother experience. Performance is another key factor - by generating the necessary files within the core, we can reuse the evaluated markdown. For instance, I don't think vitepress-plugin-llms currently supports features like dynamic routes, markdown inclusion, or snippet imports.

brc-dd avatar Apr 11 '25 09:04 brc-dd

having this feature in the core would open the door to additional plugins that can use llms.txt (e.g. semantic docs search).

S1M0N38 avatar Apr 11 '25 09:04 S1M0N38

It ultimately depends on how many users need that feature. Enabling llms: true in vitepress config is significantly more convenient for end-users compared to installing and setting up a separate plugin. The same applies to other popular vitepress features like mathjax/katex support, local search, sitemap generation - and we will likely bring back auto-sidebar mode as well. While all of these could be (and were/are) achieved with third-party plugins, the built-in options offer a much smoother experience. Performance is another key factor - by generating the necessary files within the core, we can reuse the evaluated markdown. For instance, I don't think vitepress-plugin-llms currently supports features like dynamic routes, markdown inclusion, or snippet imports.

Without plugin:

llms: true

With:

// pnpm i vitepress-plugin-llms -D
import llmstxt from 'vitepress-plugin-llms'
...
vite: { plugins: [llmstxt()] }

Not so much compicated

Besides, I don't want a month of work on the plugin (which is already used by popular websites) to go to waste...

okineadev avatar Apr 11 '25 09:04 okineadev

I would like to maintain the llms.txt generation functionality

okineadev avatar Apr 11 '25 09:04 okineadev