Add support for offset-based pagination to crate versions endpoint

[eth3lbert]

This PR add support for the offset-based pagination to crate versions endpoint, which is a prerequisite for maintaining consistent pagination layout in the frontend.

[syphar]

btw, thanks for your library, I'm currently in the process of migrating docs.rs to it :)

The reason I'm opening this issue is because https://github.com/rust-lang/rust/issues/43165#issuecomment-1416781764 points out that TOML syntax highlighting is already available on the details pages for crates. It doesn't seem like it would take much to inject it into the HTML for the docs themselves.

We have some pages where we actually do highlighting, there are:

• the mentioned crate detail page ( = the markdown readme): https://docs.rs/crate/sqlx/latest
• our own source pages (where just show the crate sources): https://docs.rs/crate/sqlx/latest/source/Cargo.toml

in both cases we have the actual source and render a highlighted HTML version using comrak.

Now back to your topic. Docs.rs just runs a (relatively) normal cargo doc command and stores the final HTML. So to add highlighting to these documentation pages we would have to parse the old HTML, find code blocks, and rewrite these with a highlighted version, and all taking into account the styling outside of the code block.

So while we actually could rewrite HTML (we are already rewriting small parts, for example to add the top / bottom bar), the actual correct place would be rustdoc, since then users using docs.rs would see the same docs as users running the doc build locally. Also I believe a well readable doc page without broken styling, across all nightly rustdoc versions would be really hard.

[syphar]

of course we could also use frontend techniques to do something similar, but with similar issues keeping it stable across versions

[GuillaumeGomez]

I think we should just add highlightjs at this point and run it by default on all pages.

[syphar]

I think we should just add highlightjs at this point and run it by default on all pages.

you mean that rustdoc should do this ? or docs.rs?

I don't have much experience with highlight.js, I only know that for the pages we render ourselves (like our source) it was replaced by server side highlighting.

Wouldn't this conflict with the cases where rustdoc is already highlighting, like its own source pages?

[GuillaumeGomez]

I meant docs.rs. And I can write the JS so highlightjs only takes a look at not highlighted code examples.