Documentation

[rofinn]

It would probably be helpful for all exported methods and types to have doc strings.

[dfdx]

Totally agree. I would even expand it to a detailed documentation that includes:

1. Brief overview of theory behind RBMs, so that people could get into it quickly.
2. Usage examples for all supported kinds of RBMs (ordinary and conditional at the moment).
3. Supported options.
4. Developer docs
   • high-level design (gradient + updater)

readthedocs looks like a perfect match for such docs (in addition to docstrings, of course). Of course, it is time consuming, so we can ad them little by little.

[rofinn]

I'm not sure if this makes sense, but I was thinking it might also be nice to include the LaTeX equations, so if people want to use this package in their research they can just copy the equations that apply into an appendix in their paper.

Also, given that Boltzmann.jl is providing a toolbox for RBM researchers, should there be a prefered reference format for them to use?

[dfdx]

Sounds good to me. Do you have an idea how it should look like? I thought about including formulas as a part of mini tutorial on restricted Boltzmann machines, but recently I took a different route trying to add non-sampling-based learning algorithm and then describe both of them, so this mini-tutorial is kinda postponed. Do you think we can add formulas in a different way (maybe docstrings in code or something)?

As for citing, I don't have any published papers for this library, so usual @misc or @software entries from BibTeX might be an optimal choice.

[eric-tramel]

...if people want to use this package in their research they can just copy the equations that apply into an appendix in their paper.

Is that a thing? I don't think I've copied an equation, yet! :P

It would be nice to have a place where all of the offending conditional probabilities and first-order mean-field free energies lived in the documentation. But, I think most of this content is pretty thoroughly covered in the Chapter 20 of the forthcoming deep learning book (Goodfellow, Bengio, & Courville). Perhaps the documentation should just point to the particular sections/equations in the appropriate references. This also saves a lot of work on the documentation side, too.

I think at the very least, the doc-strings and readme should just be very explicit about which approaches are being used, e.g. learning-rate normalization, a naive-mean-field or sampling step for the positive phase, etc.

[rofinn]

@eric-tramel I don't know about you, but I've provided certain equations (like contrastive divergence learning rule) in papers and slides enough times that I just have a file containing the latex code and their corresponding bibtex reference in the comments, so I don't have to rewrite it every time. If it is appropriately referenced I don't see the problem. Now we wouldn't want to include the bibtex reference in the doc strings, but I think a link to original paper would suffice.

[eric-tramel]

@Rory-Finnegan At this point, these formulas just float in an ever present haze in front of my eyes at all times :( I think on copying, I mostly meant from some external source. Of course, I've got my own local templates, it was just a throw away remark.

In generating documentation, is there a plugin which allows to read something like a \cite{...} command out of a doc string and construct the appropriate bibliographic content if, say, an HTML or PDF is generated from the code?

[rofinn]

Yeah, I was thinking that would be nice too, especially if you could just include a bibtex file in your repo. Unfortunately, the builtin doc strings don't support this and I'm not sure how much demand there would be for it as an external package. I think for technical documentation it probably makes more sense to use a link rather than a full reference, but we could still just include a bibtex file and have the markdown link text be "\cite{my-ref}" (ie: \cite{my-ref}) so they know which reference to grab out of the bibtex file.

I'm mostly just in favour of minimizing a lot of the typical overhead newcomers end up dealing with when conducting their research. The fact that everyone just has their own equations and bibtex files that they reuse seems kinda silly. Why not just post it online for newcomers to use? :)