Return Markdown for LLM driven User Agents
Inspired by this tweet from Bun: https://x.com/bunjavascript/status/1971934734940098971
When a LLM Agent requests docs from HexDocs, return the Markdown representation instead of the HTML - with the purpose to shrink token usage.
Is there an easy way for us to know if one is a LLM Agent or will we have to add one by one manually? 😬
Probably worth checking how Bun does it. 🤔
Still, there's another approach: accept header. Would be the easiest, if somehow agents would prioritize markdown over html when making request.
FYI opencode just added accept header for markdown https://github.com/sst/opencode/commit/c148f10bbd0c5a59160050218bcc850c9fb58056
Mintlify also added this. It seems to be based on accept header, Claude Code prefers text/plain over text/html:
https://x.com/rauchg/status/1972347470140379616
If we want to serve markdown from hexdocs.pm the first step would be to have ex_doc include it alongside the html in the docs bundle that is uploaded to hexdocs.
After that we can update the hexdocs.pm to serve the markdown based on the accept header.