Tracking documentation coverage?

[thesamovar]

After creating #1080, I was thinking about how we could catch issues like this (i.e. a feature that is not documented anywhere). For our test suite, we have a notion of code coverage. While not perfect, this catches a lot of stuff. I'm wondering if there is something similar we could do for documentation? One could imagine a tool that attempts to automatically work out which bit of documentation applies to which bit of code, and gives you a log of what it thinks is missing (which you could manually fix by adding in comments to both the code and docs).

[thesamovar]

It would be nice also to have it be aware of the difference between user documentation, advanced and reference documentation.

[mstimberg]

I think this could be useful, but we should not overthink it, either. For example, I think it could give you a list of non-covered think, which you could scan for things that look important – I don't think we necessarily need a syntax for excluding things from that list.

There are also two somewhat separate things:

1. The reference documentation should cover all methods/attributes. If it does not, it means that there is a docstring missing. We can check for this with eg. pylint (something like pylint -d all -e missing-docstring brian2, needs some more configuration to not check brian2.tests)
2. The user documentation should mention all methods/attributes at least once. I guess we could note down all links to the reference docs as part of our "briandoc" extension?

[thesamovar]

You could have just a separate text file with a list of strings to filter out. Low syntax burden but allows you to have a tool that can report only what is missing and what you haven't already dealt with.

Combination of pylint and something that checks that everything in the reference documentation has at least one user documentation link sounds like it would work, yes!

---

From: Marcel Stimberg [email protected] Sent: Wednesday, 29 May 2019 08:35 To: brian-team/brian2 Cc: Dan Goodman; Author Subject: Re: [brian-team/brian2] Tracking documentation coverage? (#1082)

I think this could be useful, but we should not overthink it, either. For example, I think it could give you a list of non-covered think, which you could scan for things that look important – I don't think we necessarily need a syntax for excluding things from that list.

There are also two somewhat separate things:

The reference documentation should cover all methods/attributes. If it does not, it means that there is a docstring missing. We can check for this with eg. pylint (something like pylint -d all -e missing-docstring brian2, needs some more configuration to not check brian2.tests) The user documentation should mention all methods/attributes at least once. I guess we could note down all links to the reference docs as part of our "briandoc" extension?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/brian-team/brian2/issues/1082?email_source=notifications\u0026email_token=AAWE6PVZNBGCSW5UPDYJSZTPXYW4DA5CNFSM4HQBT3Z2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWOOJEY#issuecomment-496821395", "url": "https://github.com/brian-team/brian2/issues/1082?email_source=notifications\u0026email_token=AAWE6PVZNBGCSW5UPDYJSZTPXYW4DA5CNFSM4HQBT3Z2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWOOJEY#issuecomment-496821395", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[tapaswenipathak]

Hi @ thesamovar, @mstimberg: Interested, can I work on this?

[mstimberg]

Hi @tapaswenipathak if you are still interested, please feel free to work on this!