Unstable Features for faster iterations

[mockersf]

trafficstars

Rendered

[MiniaczQ]

How do you imagine CI/CD working with unstable features on the main bevy?

[alice-i-cecile]

How do you imagine CI/CD working with unstable features on the main bevy?

IMO one-experimental-feature at a time would be the strategy, for tests only. That'll increase load on our CI though, and may tip us over into "actually a priority to upgrade".

[mockersf]

How do you imagine CI/CD working with unstable features on the main bevy?

With only GitHub runners, no CI/CD for unstable features. This is actually already an issue in Bevy as we don't test non default configurations.

If we can get more capacity, we could run more feature gate combinations, but I don't think that should be blocking for merging a PR. If a PR break an unstable feature:

• the PR author may fix it if they have the possibility
• or it can be reported on the unstable feature tracking issue, then someone with more knowledge on that feature can fix it

[alice-i-cecile]

Some current PRs that would benefit from being unstable features:

• animation
• geometric primitives
• viewports
• raycasting
• Command error handling
• runtime system insertion
• window icons
• rumble support
• system graphs

I've tried to select things that I think are:

• large features
• things we probably want
• complex APIs that should be experimented on, played with and iteratively developed

There are obviously other PRs that I think should just be merged outright (or merged after a bit of cleanup), but those are a distinct category.

[james7132]

IMO anything that would introduce a new crate or engine capability (looking at animation here), should probably fit into the RFC category and go through a more rigorous design process, as much as I would like for animation to be developed and integrated faster.

Though that brings up question of ecosystem crates depending on unstable features. We should probably discourage ecosystem crates from developing stable features of their own on top of unstable bevy features.

[cart]

I think it's worth calling out what our policy is on "competing implementations". Are we going to allow merging 3 different competing unstable skeletal animation systems? What if they conflict with each other? If we only allow one, how do we pick a "winner" without going through a "full" review process (including pulling in "lead maintainers")?

I also think its worth explicitly calling out what should be a "3rd party crate" and what should be a "first party unstable feature". I personally think we should still push experimental / un-prioritized features to 3rd party crates when possible. 3rd party crates still allows for collaboration without bloating bevy internals (or bloating the bevy dev process / getting blocked on maintainer approval). It also more cleanly accommodates competing implementations, and in somes ways is a simpler user-facing conceptual model than "unstable bevy features". I think the decision tree should be "can this be expressed as a third party crate? if yes ... do that. If not (because it needs to make internal changes), use an unstable feature. Skeletal animation would probably need to be an "unstable feature" because the PBR pipeline can't (yet) easily accommodate 3rd party changes to mesh bindings (and im not convinced it should). On the other hand, "geometric primitives" and "system graphs" can be (and have been) expressed as third party crates. Maybe to help incentivize and streamline the "experiment in 3rd party crates first" process, we could have a way to "boost" 3rd party crates that are currently being considered for inclusion in bevy?

[alice-i-cecile]

On the other hand, "geometric primitives" and "system graphs" can be (and have been) expressed as third party crates. Maybe to help incentivize and streamline the "experiment in 3rd party crates first" process, we could have a way to "boost" 3rd party crates that are currently being considered for inclusion in bevy?

Boosting would be valuable. It's often hard to attract contributions to third-party crates, because it's not clear whether the effort will be wasted.

I'm also wondering whether unstable features could be a good way to start the upstreaming process, without blocking on your full review attention. bevy_mod_picking and bevy_kira_audio are both nearly ready IMO (and are obvious "this needs to be core to the engine" functionality), and are the clear "winners" within the ecosystem.

Fundamentally, the upstreaming process is likely to be rather involved, and trying to get all of the details from both a usability and code quality perspective ironed out in the initial code review is likely to be less effective, slower and more frustrating than taking a "merge and refine" approach.

[cart]

Another quick thought: as this is pitched as a way to resolve "scaling issues" and it does come with a number of tradeoffs (which are already called out), we should fully discuss alternatives first. I think @mockersf already called out the clearest alternative. Rather than creating a new class of "unstable" code, we could instead just horizontally scale merging "stable" code (ex: some type of "subject matter expert merge rights" approach). It would require a lot more care to maintain the integrity of the engine (especially from my perspective ... there have been a lot of things that many "active" community members have voted to merge that I wouldn't merge without tweaks). But I think its something we should be "growing into".

[james7132]

Another quick thought: as this is pitched as a way to resolve "scaling issues" and it does come with a number of tradeoffs (which are already called out), we should fully discuss alternatives first. I think @mockersf already called out the clearest alternative. Rather than creating a new class of "unstable" code, we could instead just horizontally scale merging "stable" code (ex: some type of "subject matter expert merge rights" approach). It would require a lot more care to maintain the integrity of the engine (especially from my perspective ... there have been a lot of things that many "active" community members have voted to merge that I wouldn't merge without tweaks). But I think its something we should be "growing into".

Just wanted to call out in this thread that there has been a sizable discussion of these kinds of alternatives in https://github.com/bevyengine/bevy/discussions/4131

[alice-i-cecile]

there have been a lot of things that many "active" community members have voted to merge that I wouldn't merge without tweaks

One of the perspectives (that I'm not sure how I personally feel about) is that we take a merge-first, tweak-later approach, especially with more folks with merge rights.

The majority of PRs we receive are clear improvements over the existing situation, and while they aren't always perfect in terms of code quality / docs / API design, collaboration is generally much easier once the code is merged in (and we'll have initial user feedback). PRs-to-PRs are a thing, but there's a lot of steps and they're not obvious: without assurance that "yes this will get merged soon" it's often hard to prioritize work done to review or improve others' PRs.