Feature request: online linkable manual

[colinbrislawn]

Good evening,

I have had a great experience using vsearch and recommend it to everyone. The documentation is clear and concise and makes vsearch very accessible. I'm happy to see your high performance implementation includes great documentation to match.

Currently, vsearch documentation is provided in man vsearch, and vsearch --help, and in the PDF download. So when I provide support to others or discuss functionally over email, my best option is to say

Try vsearch --search_exact see: https://www.drive5.com/usearch/manual/cmd_search_exact.html

Or

Try vsearch --search_exact see page 25 (PDF link)

I would love to see the manual published online so I could link to sections directly.

Is this something you are interested in pursuing?

Colin

[torognes]

I agree that having the documentation online in html format would be very useful.

Perhaps it could be automatically converted from the current man page format and into html with anchors that could be linked to. I have tested this briefly with the man2html utility (which appear somewhat old). It worked mostly ok, but did not properly handle the [step] and ~ syntax in the current version of the manual. Non-standard characters (unicode) were also not handled well.

[colinbrislawn]

Looks like there are lots of ways to make HTML from man pages. But the numerous options makes me think that none of them are perfect.

I know you and @frederic-mahe have considered using another platform for writing documentation #269. Switching to .md files and a static site generator would elegantly solve this problem, but it would require duplicate work or move information further from the binary.

[frederic-mahe]

It seems possible to display man pages on Github Pages (gihub.io). See for example https://vcftools.github.io/man_latest.html

[colinbrislawn]

I wonder how they did that. I was hoping to see they were using CI to automatically build their website from the man files, but I couldn't find the code. I presume they aren't updating their website by hand...

More methods for the man page -> html conversion. Hopefully works better than man2html

[torognes]

The website repo is here: https://github.com/vcftools/vcftools.github.io

The html version of the manual does not seem to be fully updated, as the latest version number in the man file is 0.1.17, not 0.16.1: https://github.com/vcftools/vcftools/blob/master/src/cpp/vcftools.1

Maybe they just run som kind of man -> html converter and update the website semi-manually.

The simplest would perhaps be to run groff:

groff -mandoc -Thtml vsearch.1 > vsearch.1.html

The result is here:

http://folk.uio.no/torognes/vsearch.1.html

It could be manipulated with a script and uploaded to a site automatically.

[colinbrislawn]

That groff html manual is really elegant. Would it be possible to add in more anchors throughout so we could link to specific sections, or could this be done better with separate manual pages like Fréderic suggested?

and uploaded to a site automatically.

github: publish
	ghp-import $(OUTPUTDIR)
	@git push -fq https://${GH_TOKEN}@github.com/$(TRAVIS_REPO_SLUG).git gh-pages > /dev/null	

[frederic-mahe]

I've tried roffit but it does not seem to recognize more recent groff commands (such as URL commands for instance).

groff -mandoc -Thtml vsearch.1 > vsearch.1.html produces an HTML page using ascii charset encoding, ideally it should be replaced with utf-8 charset. It also fails to parse tables. I'll try to dive into groff's documentation to see what can be done.

[colinbrislawn]

Real-world example of user finding vsearch documentation in usearch docs, because we don't have an online manual. #360

Sorry to bother you guys. Any updates on groff?

[torognes]

I have looked a bit more into this issue today. The groff command has options for adding tags to the man file giving named anchors in the html file if the macro package www is included.

I have added tags to some of the commands and options in the manual using the .TAG macro, like this:

.TAG version
.TAG v
.TP
.B \-\-version \-\-v
Output version information and a citation for the VSEARCH
publication. Show the status of the support for gzip- and
bzip2-compressed input files.

I will continue to add tags to the rest of the options. A problematic issue is that some options are mentioned several times in the document under different commands / headings.

I have also added links using the .URL macro, for example:

.PP
\fBswarm\fR, a fast and accurate amplicon clustering method by
Frédéric Mahé and Torbjørn Rognes, available at
.URL https://\:github.com/\:torognes/\:swarm "(link)"
<https://github.com/torognes/swarm>.

For wide compatibility with the man command and the PDF document, I have just added a linked "(link)" in front of the URL and kept the url in angle brackets.

See the man page for groff_www for details.

The html can then be generated with this command:

cat vsearch.1 | sed -e 's/\\-/-/g' | groff -m mandoc -m www -Thtml > vsearch_manual.html

The character sets seems to work fine. On my Mac I had to convert the unicode input into latin1 before running groff.

A testing page can be seen here:

https://folk.uio.no/torognes/vsearch_manual.html#version

I plan to continue this work and put this html file into the releases.

I also plan to see if I can make a CSS style set that looks a bit more modern.

Table formatting does not work correctly neither for pdf nor html. It is only used once, under Masking options, so it could possibly be avoided.

[colinbrislawn]

That's a really solid start. I'm not even sure if it needs fancy CSS!

I guess adding the cross links between different sections is going to be the hard part, but this is the main value-add. The goal is to be able to tell a user

you can include terminal gaps by changing the --iddef see vsearch_manual.html#iddef"

So... could we programmatically include <a name="flag"> anchor for each input flag? All input flags are unique, so this should work really well for html anchors!

If there is interest in hosting this as a GitHub pages site like torognes.github.io/vsearch, I can help get that set up. Hosting it on your university website also works great.