links from manuals to source code

[fricklerhandwerk]

the source code is the single source of truth, and oftentimes the only one. manuals should link to the source code, where appropriate, for better discoverability of implementation.

it should help advanced users solve problems and ease onboarding of potential contributors.

• proposal:
  • use GitHub URLs to commit hashes and specific lines until further notice
    • commits may be different and have to be updated manually for changes
      • still better than pointing to code that changed
    • example and prior discussion: https://github.com/NixOS/nixpkgs/pull/121015
  • build infrastructure to use custom syntax such as ./path/to/file#L123-456
    • render HTML to convert to GitHub URL pointing to commit hash of built revision
    • check for correctness
      • at least that files and lines actually exist
      • even better compare contents between builds somehow

[roberth]

To me, linking the source is like giving up. This is a technical non-solution to the social problem of contributors and maintainers not thinking about communicating their work.

Linking all the right code and maintaining the correct line numbers takes a lot of effort that would be better spent on actual documentation that addresses user needs and teaching contributors how to find code themselves. Teach a man to fish.

One small trick to help with this, is to remove all user-focused inline comments from scripts and expressions and replace them by links to the relevant documentation files. This solves the root cause of the problem:

the source code is the single source of truth, and oftentimes the only one.

Please consider how other serious projects implement documentation. Why don't docker or postgres or git link to their sources? I'd say Nix is contrarian enough as it is, and we don't need to emphasize experimentation in the area of documentation. We just need to catch up to the standard.

should help advanced users solve problems

Advanced users will already have developed the skill to navigate a code base.

ease onboarding of potential contributors.

There's some potential for this, which is why I'm not entirely opposed to this idea. We already link NixOS modules in our documentation and we could do the same for packages and lib functions. Crucially these don't bitrot because they're extracted from the source. The remaining areas are stdenv, hooks and other logic, which I think are too important not to document them properly.

I'm not going to stop anyone from implementing the technical solutions suggested in this issue, but I will ask them to reflect critically whether they're trying to be effective at improving documentation, or whether they're implementing a passion project, which may be ok.

[fricklerhandwerk]

Very good points, thank you.

documentation that addresses user needs and teaching contributors how to find code themselves

Yes. We're actively working on documentation directly addressing user needs. That is outside of this issue's scope though.

My idea to teach contributors how to find code themselves is to provide a clear and obvious way into the code. There are different approaches to this in general. Links are the most obvious thing I came up with, and we can add them right now. Annotating the source tree in the README is another one.

I fully agree that it is not maintainable, and eventually all these things should be derived from the source, not the other way round. But we are resource constrained, and just writing a few things down in a central place already helps, quickly and measurably. It does not have to scale from the start, we just need something helpful at all. Patterns will emerge that can be automated.

I'd say Nix is contrarian enough as it is, and we don't need to emphasize experimentation in the area of documentation. We just need to catch up to the standard.

I don't think this is about being contrarian or novel, just about delivering solutions that are in demand. Fully agree that catching up to the standard is the way to go, but we need something to work with while on the way there.

remove all user-focused inline comments from scripts and expressions and replace them by links to the relevant documentation files

I disagree with this specifically, because then you won't have these comments next to the code any more. As long as we don't have a tool from science-fiction where all the data is seamlessly accessible through one homogeneous user interface, we have to place bits of information that belong together next to each other. I'd rather render the comments into a web representation and link back to the source, exactly as we do with nixpkgs and NixOS modules already (as you say yourself).

Advanced users will already have developed the skill to navigate a code base.

We could instead give people an easy way in that does not require them to spend precious time on developing yet another arbitrary skill.

trying to be effective at improving documentation

That's what we do. Therefore, thanks again for your input and for helping to keep focus.

[roberth]

Thank you for addressing most of my concerns. I will remain skeptical of technical solutions.

but we need something to work with while on the way there.

Don't underestimate the effectiveness of "just doing the work": it's clear what the goal is and it gets the job truly done.

remove all user-focused inline comments from scripts and expressions

I understand your concern, and I agree that having everything close together in functional units is much favorable over solutions pull functional units apart until multiple locations separated by technical concerns. When judging whether a comment is user- or implementation focused, it generally doesn't hurt to err a bit on the side of keeping some of it in the code.

a tool from science-fiction where all the data is seamlessly accessible through one homogeneous user interface

hyperlinks

Even the unidirectional ones are really good, because they focus attention towards the good and useful parts of our "web".

spend precious time on developing yet another arbitrary skill.

Being effective at grepping through a codebase is a fundamental skill that's highly underappreciated.

That's what we do. Therefore, thanks again

Awesome, and I appreciate even more the effort you are putting in. I have to be on the sideline now, cheering you on, but I'm glad we could take the time to discuss this.

[fricklerhandwerk]

a technical non-solution to the social problem of contributors and maintainers not [...] communicating their work

I absolutely agree it is primarily a social problem. We have multiple things in the making to address that (see meeting notes), and I'm happy about more suggestions we can put into practice. Ultimately I think it's purely a question limited resources, and the most effective thing we could do is have the Foundation (or community) fund a full time maintainer. What I'm currently doing is the next best thing, funded exclusively by Tweag.

[toraritte]

+1 for linking source

I do agree with @roberth that

1. linking to source in lieu of maintaining proper docs should be a no go - except if it used to add entries incrementally as @asymmetric suggested and/or if the design choice for a documentation system is to always link (e.g., see map in the Haskell docs)

2. it is an extra layer of complexity to implement and maintain (probably; how does the Haskell community do it?)

3. learning the skills to navigate code bases is an invaluable skill

However, linking to source respects the time and effort one deems to put in to improve the docs, because right now, one not only has to find the code relevant to the functionality they are looking for, but also to find the code to the documentation itself. If that person is not a regular in the code base, a week is enough to forget everything, and next time they might just skip the hassle (or, they simply can't afford to spend time on it).

Cases in point from personal experience: patchShebangs and builtins.fetchGit - never regretted putting in the time, but even though the latter was written 2 weeks ago, I would probably have to redo the whole research if I had to verify something.

As mentioned above, the Haskell community opted for including the link to source as a design choice. It's another non-trivial code base, and if there is a function that is not documented, the source would be linked, and one could jump in to remedy that. (A system like that would also have a side effect that any new functionality would appear in the docs as a stub, immediately drawing attention if it is undocumented.)

[roberth]

the Haskell community

In Haskell, this is easily achievable. In Nix land, we don't by default have a namespace where our function definitions map to. Nonetheless nixdoc can already do it, although it requires some help from Nixpkgs via locations.xml. It works: https://nixos.org/manual/nixpkgs/stable/#chap-functions (though locations may be out of date until the manual redeploys, fwiw).

community opted for including the link to source as a design choice.

This is not true. It was implemented by the original haddock author and I doubt that the feature was even discussed with the community, which was tiny and very different at that time.

More significantly, Haskell libraries are notorious for having sparse documentation.

as a design choice

The GHC manual, which does not rely on haddock, is a pretty good reference work and does not link to its implementation. If this was a community-wide design choice, wouldn't GHC link to its source?

immediately drawing attention if it is undocumented

Not immediately.

---

I'm not opposed to haddock-style links in reference documentation like we already have, and I'm not opposed to the development of tooling to support this for, say, bash functions. However, linking from arbitrary parts of the manual to the source may not be a net positive because of the opportunity cost and allowing a bad practice.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/summer-of-nix-documentation-stream/20351/2

[toraritte]

Thank you for correcting my assumptions regarding haddock and for linking nixdoc because I missed it so far!