Do we need a place to document experimental features?

[tomchor]

I was wondering if it would be advantageous to have a place dedicated to document the features that are still experimental. I think that'd make it easier for other people to contribute to those features and having them mature faster.

The main motivation for me to ask this is that often users (myself included) will ask about a particular feature that's still under development (generally immersed solids or parallelism), and someone ends up describing the state of affairs at that point and pointing to a validation script and the user has to figure everything out from there. I realize all these things are available in issues and PRs, but it's hard to find the right ones and distill what's "relevant" information from there.

I'm going through that this moment with ImmersedBoundaryGrid, which I'm trying to advance. I can work backwards from a given script to figure out how the implementation exists now and get a sense of to expand it, but it'd be way easier if I could read about what's implemented already, why that's the chosen implementation, bumps on the road, etc.

Another motivation is that the code has grown so much lately that I sometimes see PRs/issues here about some features and I realize I simply don't recognize (and I'm fairly involved with Oceananigans' development). Some of those features might be useful for my research, or they may be things that I'd be interest in developing that I'm missing out on.

Possible places to host that info would be a dedicated section of the docs possibly named "Experimental features", but maybe the easiest way would be allowing a wiki for that. I think discussions wouldn't be as organized.

Thoughts?

[glwagner]

How about discussions? Anyone can start a discussion about running simulations with immersed boundaries, for example. Perhaps that will also encourage developers to open discussions like that too.

Wiki and docs are more work because they will have to be updated / changed. That’s why experimental things aren’t well-documented; it’s an onerous burden to formally document rapidly changing features.

[navidcy]

Yeap, I agree with @glwagner.

In principle @tomchor that's a great idea! But in practice I'm not sure we have the capacity to do so at this point. I'd be keen to see a section with "experimental" features but personally I don't know if I have the time to put in documenting things that change by the week...

However, experimental features usually do come with a docstring. So people living on the bleeding edge of Oceananigans.jl at least know what arguments need to provide, etc. That's something! Perhaps at first stage we make sure that all functionality has some sort of minimal docstring?

[tomchor]

Just to be clear, I'm not suggesting that we document experimental features to the level that we document mature ones. That'd be impossibly hard to maintain.

But maybe a page somewhere with a list of experimental features (all I know about are immersed boundaries and parallelism, although I'm sure there are more that I'm not aware) that links to the relevant issues/PRs/discussions and maybe to relevant files in src? Otherwise I think it's hard for users to know what experimental features there are since they have to browse issues/PRs/discussions looking for stuff in the titles.

[glwagner]

What if we open a discussion any time we are planning to develop some major new experimental feature, and then add to the discussion as new info and PRs come in? We can make a new tag "experimental feature" so that you can search for all relevant discussions.

[glwagner]

That way if users decide to experiment with a new feature they can also contribute their experience to the discussion. It might be more fluid and easier for everyone to work with than something more formal like a wiki?

[tomchor]

That way if users decide to experiment with a new feature they can also contribute their experience to the discussion. It might be more fluid and easier for everyone to work with than something more formal like a wiki?

I don't really see the wiki as being formal, but I do agree that discussions are more low-maintenance. I think if we can tag them properly as "experimental features" it would really help.

[tomchor]

I took the liberty of creating this label and applying it to discussion https://github.com/CliMA/Oceananigans.jl/discussions/2345

I think if we tag these discussion appropriately so that it's easy for users to find them, I'd be okay closing this issue. Maybe we can see how it goes for a couple of weeks and if we feel good about it, mention it on the docs?

[glwagner]

We have an "experimental features" part of the discussions now