[Proposal] Community Strategy for this organization

[choldgraf]

tl;dr

We need to create a strategy for the EBP organization and community moving forward, as we are nearing the final year of the grant that funded this project, and must decide how it will sustain itself moving forward. This is a proposal to convert the project into an open and inclusive community modeled after other major communities in this ecosystem.

• Find the proposal text here
• Provide comments and discussion in this issue, OR:
• Provide discussion in sub-threads in this GitHub Discussion

Background

For the last ~1.5 years, the Executable Books Project (EBP) has operated as a team of collaborators funded off of a single grant led by John S. (@jstac), with co-PIs Chris H. (@choldgraf) and Greg C.(@gregcaporaso). This has resulted in development across several repositories in the executablebooks/ GitHub organization. We have followed informal practices as a small and independent team.

We are entering the final year of the grant, and assume we won't have the same dedicated funding in the future. We must decide how we'd like this project to evolve, and what steps it will take to get there. If we wish for the project to sustain and grow its impact, we must develop a strategy for ensuring the sustainability of the project after this initial grant-funded period.

This community strategy document

I've written up a document that describes some major strategic areas where we will need to develop in order to professionalize our community and abide by best organizational practices. I've written this up in the following document:

https://hackmd.io/@choldgraf/HkgAIHtQY

It is structured around these major strategic areas of organization / community:

• define the mission and values of the organization
• define our governance and decision-making process
• define our code of conduct and mechanisms for its enforcement
• define some community practices
• describe our organizational designation

Together, these major areas describe the transformation of the Executable Books Project into the Executable Books Community (referred-to below as Executable Books, or EB).

Comments?

I'm posting this document for comment from the broader community. Especially the @executablebooks/ebpteam and @executablebooks/web-team and any regular contributors across the repositories of https://github.com/executablebooks thus far. A few that come to mind that may not be in the groups linked above: @astrojuanlu @stevejpurves @ericholscher @humitos @emdupre @martinagvilas @kirstiejane @malvikasharan @firasm @fperez @consideRatio @damianavila @moorepants @minrk @akhmerov @foster999 (and others feel free to tag folks if you think they have a stake in this project :-) )

I would like to know what others think about the major ideas in the document shared above. It is intentionally vague in some areas (where we must put in work to create specific proposals), and intentionally specific in other areas.

I've also created a Discussion thread here, with sub-threads for each of the major areas described in this proposal. I invite comment either in this issue, or in that Discussion thread if there are comments specific to a section.

Some questions to ask

• does this proposal seem sustainable and growable ?
• will it create the kind of community that you'd enjoy participating in?
• does it seem heavily biased towards a particular kind of person or perspective?
• are there any red flags in there? Anything crucial we are missing?

Next steps

I'd propose that we do the following:

1. Iterate on this document and improve it as necessary, addressing suggestions and objections from Contributors.
2. Propose a PR to executablebooks.org that incorporates these ideas into our meta docs
3. Decide whether to adopt this proposal, via a vote of the newly-designated Steering Committee, and via merging this PR (or not).
4. Begin implementing the necessary next steps to adopt these practices (e.g., define an RFC process, formalize a code of conduct, formalize team policies, decide on an organizational home, etc).

Our goal should be to have a well-functioning community that adheres to the principles described above, and with an organizational structure that is explicit. Our final "deliverable" as PIs of the grant should be to relinquish control of the project to this new organization and allow it lead the Executable Books community into the future.

Tasks

Team practices

• [x] https://github.com/2i2c-org/pilot/issues/105
• [ ] https://github.com/executablebooks/meta/issues/452
• [x] https://github.com/executablebooks/meta/issues/441
• [x] https://github.com/executablebooks/meta/issues/375
• [ ] https://github.com/executablebooks/meta/issues/112
• [ ] https://github.com/executablebooks/meta/issues/87
• [ ] https://github.com/executablebooks/meta/issues/79
• [ ] https://github.com/executablebooks/meta/issues/50

Governance

• [ ] https://github.com/executablebooks/team-compass/issues/3

Organizational home

• [ ] Reach out to folks in the Jupyter community to understand if there is interest in the Executable Books community becoming a sub-project

[mmcky]

thanks @choldgraf for putting this together and initiating this discussion. I think it is a great time to open up this discussion so we can aim to continue developing the community over the coming year.

I will have a close look at the documents over the next few days.

Perhaps we should aim to review the documents and provide feedback prior to the next Monthly team meeting scheduled for October 6th (USA, UK)/ 7th (Australia) and allocate some time to review and discuss that feedback at that meeting.

[astrojuanlu]

Thanks a lot for the ping @choldgraf ! I gave a first pass at the document and it looks great 🙌🏽 The biggest questions from my side are

(a) are there other Jupyter sub-projects that are working in a similar fashion as EB? (I don't know if JupyterLab or JupyterHub have ad-hoc governance structures and processes) (b) how much of the document holds if EB effectively becomes a Jupyter sub-project? (c) since EB becoming a Jupyter sub-project is a very likely scenario, wouldn't it be more efficient to lay out the processes assuming this would be the case? or, alternatively (if you consider that there might not be up-front consensus on this matter), outlining what drawbacks would this choice have?

[choldgraf]

hey @astrojuanlu - just some quick responses because these are all good points.

In general, I tried to describe structures that we'll need to grapple with regardless of whether this is a standalone project, or part of Jupyter. Some details below:

(a) are there other Jupyter sub-projects that are working in a similar fashion as EB? (I don't know if JupyterLab or JupyterHub have ad-hoc governance structures and processes)

Most Jupyter sub-projects have a notion of a "core team" (for example, the JupyterHub core team). There's some governance refactoring going on, and one part of this will be that each "official" sub-project needs to define the equivalent of a steering council (they can call it what they like, but the project needs to have an official mechanism for deciding, and group of people who decide). If we wished, I think that the EB "steering council" would naturally become this decision-making body within a sub-project of Jupyter.

(b) how much of the document holds if EB effectively becomes a Jupyter sub-project?

I think that we'll need to work out most of these structures regardless. The biggest difference is that if it were a Jupyter sub-project, we would inherit some stuff that we would otherwise need to create ourselves. The biggest would be a Code of Conduct and a legal home for the project. (as mentioned above, I think the governance and team policy stuff we will need to work out regardless).

(c) since EB becoming a Jupyter sub-project is a very likely scenario, wouldn't it be more efficient to lay out the processes assuming this would be the case? or, alternatively (if you consider that there might not be up-front consensus on this matter), outlining what drawbacks would this choice have?

I flip-flopped on how much to focus this document on that scenario. My thinking here is that we should get alignment on the kind of community goals we have for this community, and then decide if Jupyter fits within that vision. I think that it does, but I'm not sure what others think :-) But I think it's important to discuss and think about this stuff either way, rather than just saying "oh we don't need to think about a code of conduct because we just inherit the Jupyter one". It's still important to agree that we need a CoC that is chosen through a community decision, even if we end up adopting a pre-existing one. Does that make sense?

(also I don't know that it's "very likely" or not haha - we'd still need to ask the Jupyter community if this is of interest to them! so I don't want to assume that is a foregone conclusion)

@mmcky that sounds like a good plan - I don't think there is a particular "hurry" on this document, though I do think it will take time and discussion to take steps in the directions that are proposed there which is why I decided to open up this question now rather than at the 11th hour when the grant is about to run out :-)

[jstac]

Very nice work @choldgraf , I fully support the views set out in the document so far.

My brain has increasingly shifted to a lazy evaluation model: only use CPU cycles on issues as the need eventuates. Hence I like @astrojuanlu's take. For example,

The biggest difference is that if it were a Jupyter sub-project, we would inherit some stuff that we would otherwise need to create ourselves.

So perhaps, for that stuff, which is quite a lot of stuff, the discussion should be binary --- are we happy to inherit it or not?

(For me the answer is yes.)

[firasm]

Thanks @choldgraf ! As a mere "user" of the project, I feel honoured to be included 😇. Though, I am quite heavily invested as over 800 students across ~5 courses all rely on this every year. I also support everything else in the proposal! I may add more minor points in discussions in the appropriate sections as time permits (things like +1 Rust RFC ✅ etc).

I have added a discussion comment here about the types of contributors we might expect.

On another note, like others, I am also inclined towards becoming a Jupyter sub-project. So to answer @jstac 's question, for me also the answer is yes (to inherit Jupyter stuff).

Perhaps the only thing I would bring up for consideration is if/whether the EB project might want to grow "beyond" Jupyter and become an entity in and of itself, something like Pressbooks.

Don't get me wrong, I'm the biggest fan of the Jupyter project, but the ONLY "disadvantage" I can think of being under Jupyter is that JB is seen as "a tool for computer scientists", not accessible to the general population. Then again, if this became an issue, a simple rebranding might be the cleanest solution (EB instead of JB for eg).

[chrisjsewell]

Cheers @choldgraf; we have just a long and rambling conversation lol, including touching on this. So he knows already a lot of my thoughts. But just to emphasis, I think becoming a Jupyter sub-project is strategically very important

[choldgraf]

Just a quick thought here on the Jupyter question. I thought about it a little bit and I think these are some more specific things we'd need to do in order to take that path:

1. Understand what we'd be inheriting if we became a Jupyter sub-project. I think that this is a combination of:

   • Jupyter's Code of Conduct and enforcement mechanisms for that CoC.
   • Jupyter's governing principles which are in a transition period right now, but will probably be some combination of this sub-project decision-making guide and using the Jupyter Enhancement Proposal process.

2. Decide whether it makes sense to move all of the executablebooks projects inside of Jupyter, or if only a subset make sense to move.

   • Here are things that I think obviously would be in-scope for Jupyter: Jupyter Book, MyST-NB stuff, Jupyter Cache
   • Things that might be in-scope for Jupyter: MyST Parser stuff (especially if we could build more MyST bridges with Jupyter).
   • Things that might not be obviously in-scope for Jupyter, so we'd either make a case for why they should be included in a "jupyter sub-project", or decide a different organizational home to keep them in: the base markdown stuff like markdown-it-py or mdformat, stuff that is specific to Sphinx but isn't related to execution (e.g., the Book Theme, sphinx-copybutton, sphinx-design, etc).

3. Make a JEP for the Jupyter community that proposes the incorporation of Executable Books as a sub-project. Here is an example of the JEP to incorporate Voila. We'd basically need to fill out the same stuff.

   • I think our biggest challenge here would be showing that the project has development energy that will keep it moving forward beyond the grant that we currently have. @chrisjsewell has heroically built most of the codebase in the stack, with @mmcky and @AakashGfude and myself doing the rest of the core tools work, but we are all funded on the grant and I think we will need to demonstrate that there is a path to building a broader and sustainable developer-base. (I think that we can make this case, just saying this will probably be a question point from others).

4. The Jupyter Steering Council would vote on this proposal, and if accepted, we could then need to implement any steps needed to make it official (e.g., making a change to our meta/ repository to denote the new status, writing a blog post, and moving repositories around if we decided that was the best path forward.

I think that the trickiest bit would be number 2: my personal preference is that there's a lot of value in having this stack under one roof, provided we can accommodate the different kinds of users that might contribute to different parts (e.g. the kind of person that will contribute to markdown-it-py is probably different from the person that would contribute to jupyter-book). That said, I could also see value in creating an organization dedicated to something like markdown-it-py, perhaps it would be easier to grow a dedicated community around it this way?

Does that make sense for a quick summary? I would love to know what others think about these steps.

[gregcaporaso]

@choldgraf, thanks so much for putting this together.

I like this proposal very much, and I hope to take another pass in the next few days to make sure I got everything, and to follow some of the links you included. I didn't notice any red flags or anything missing, and I like your plans for next steps.

Incorporating as a Jupyter sub-project seems like a good decision to me. I like the idea of inheriting some of the processes/content from Jupyter, and would be happy to review and think this through for specific processes/content.

[mmcky]

thanks @choldgraf

Decide whether it makes sense to move all of the executablebooks projects inside of Jupyter, or if only a subset make sense to move.

It is always hard to categorise things as there are always boundary decisions that need to be made. I tend to think that the executablebooks GitHub organisation is pretty well categorised at the moment, containing a collection of core and non-core extension style packages. I tend to find having the full stack in one organisation helpful as it makes it easier to share infrastructure, repo secrets etc. Just a thought but perhaps a good guiding principle on the location of repos would be if we think a core community can form around a collection of repos, then that seems to align nicely with a github organisation? But we would need to actively ensure it isn't perceived as a closed silo and encourage linkages with other jupyter projects.

However I fully acknowledge this comes from someone who has worked with the current structure so there is some bias there :-)

[choldgraf]

Thanks @gregcaporaso for the encouragement! I am happy to iterate content if you have any ideas for how to improve it!

@mmcky I agree with your assessment! One challenge that I see with this community is that the projects we work on are focused on a use-case instead of a particular tool. For this reason, I think we have the extra challenge that the stack in this org spans a lot of different skillsets. For example, we have web frontend / JS code, Python code in general as well as code specifically for Sphinx, and more low-level code in the MyST parsing world. I suspect that there will be different populations of people that would be able to work on these different subsets. But that may still be reason to keep things together, so long as we can build in mechanisms to coordinate between parts of the ecosystem.

[rowanc1]

Thanks again @choldgraf for writing this up (coming back to it again a few months later!), I think that the attention to strategy and governance of our community around executable-books/jupyter-book/myst is critical as we grow and start to span different communities and languages.

I am revisiting this thread in the context of working on the javascript implementation of MyST[^1]. To be consistent with the python implementation we need to start looking to standardize the tests, ASTs, and HTML output (e.g. for themes and semantics). These will require introducing standards and test-suites that can be maintained across different languages and repositories.

I am finding that I am now in need of some enhancement-proposal capabilities in our community!

I have created an issue here that discusses my needs with some questions, and am wondering how I can bring this community enhancement proposal process into reality?

My proposal is that there should be multiple enhancement tracks (e.g. myst, jupyter-book). With an example of scope being similar to changing the table of contents or discussions on introducing new syntax in myst (e.g. for citations). These enhancement proposals would be about breaking changes, introducing/evolving myst syntax, or changes that impact multiple repositories/languages at once. Things that change "MyST as a spec" fall into this category, improving core aspects of thebe or nbcache probably can be left to issues and pull requests.

These enhancement proposals would likely also stay within our community, i.e. they are not JEPs -- and are kept to our community/governance model/steering council/maintainers outlined above. I think this is true regardless of our end state in terms of umbrella governance.

I am happy to take on some of the initial work to setup this enhancement proposal process, copying language and process in the linked examples provided in @choldgraf's document.

Let me know if this would be helpful/useful/appropriate timing for introducing this!

[^1]: The javascript version will open up new workflows, publishing pipelines (e.g. beyond sphinx), and access to new communities of people (e.g. unified, mdast, mdx, react, etc.), as well as execution directly in a browser.

[chrisjsewell]

am finding that I am now in need of some enhancement-proposal capabilities in our community!

Heya @rowanc1, I'd note that I already created https://github.com/executablebooks/extension-proposals, when we were discussing this before.

If you want to start, building that out? (obviously can give you admin access)