Editor data model

[djeedai]

RENDERED

This RFC is for the (upcoming) Bevy Editor, and has little to no impact on the Bevy runtime itself (existing code).

This RFC describes the data model of the Editor, that is the representation of the Game data and the API to manipulate it that the Editor uses. The API heavily makes use of the Reflect API of Bevy to manipulate type erased objects.

The data model aims at having the following properties:

• The data mutations are diff-based instead of allowing direct object mutations. This allows all systems of the Editor to access a unique source of truth, with the added benefits of allowing to build unified undo/redo and copy/paste systems.
• The data model is an abstraction of the game data, which does not require a direct dependency on the Game code (no Cargo reference), and therefore avoids the need to rebuild or even restart the Editor process each time the Game code changes.
• The data representation can be serialized to disk and deserialized back (round-trip) without loss of data.

The details and mechanisms by which:

• this data model is effectively serialized (via reflection)
• the Game and Editor processes actually communicate (inter-process communication; IPC)
• the editing data is baked into its final shipping-ready form (asset processing)

are all outside the scope of this RFC.

[Nilirad]

Can you please put a RENDERED link at the top of your message, pointing to the .md file? 🙂

[djeedai]

Can you please put a RENDERED link at the top of your message, pointing to the .md file? 🙂

Sorry I saw this was common practice but then I forgot. 😅

[Nilirad]

The file name has still the 301 “status code”, I think you should rename it to 62. 🙂

[djeedai]

The file name has still the 301 “status code”, I think you should rename it to 62. 🙂

I picked a large number on purpose because it's for the Editor. Not sure what's the usual convention, and/or if we want a different one for the Editor (which will be the first official Bevy executable) or if we keep those mixed with Bevy (the library crates)?

[mockersf]

the number is for tracking, and it's the PR number

[djeedai]

the number is for tracking, and it's the PR number

Ok so should I rename the file then?

[alice-i-cecile]

Yes please :)

[djeedai]

Yes please :)

Done.

[djeedai]

Thanks for the review @Nilirad, I'll have a look in details and address the comments. In the meantime, I wanted to react to this:

I strongly recommend you to use semantic line breaks, as they help a lot the reviewing process. Single-line paragraphs are not very ergonomic, and make it difficult to pinpoint an issue. Start by giving the entire line to each sentence, then if you feel like it, break sentences into their logical pieces. See sembr.org for more about semantic line breaks.

This is the first time I hear about this, so I've looked at that website with curiosity and interest. Unfortunately, that felt quite short of my expectations. So forgive me as I'm going to go on an off-topic rant against it: that "semantic line break" seems purely detrimental to me. Can you clarify why you think "they help a lot the reviewing process"? For me:

• The reason I think we have a RENDERED at the top is so people can review on the final rendered text. Most IDEs, even online, have a preview mode. Maybe that's a personal preference, but I don't read/review markdown from the source; the very point of markdown is so we can have a rendered text with paragraphs and sentences that I can read like a regular book. Now you may argue you review from sources, and that's a fair argument but...

• sembr.org goes into an illustrative example about adding line breaks to increase clarity. What they really do here is break lines so they fit into the screen space. Unfortunately for them, on mobile (where I looked at the link first) even the last block with the semantic breaks extends past the visible area (without wrapping!) and so has an horizontal scrollbar. Result: none of them are more readable than the other, as all of them are truncated without wrapping, and the example falls short of demonstrating anything. And there is no other explanation given as to why semantic line breaks would be beneficial; the example was supposed to be enough. What I personally get from it is that you should have text wrapping, or maybe do wrapping by hand if you're not sure the reader will have an IDE doing it. But their example shows doing it semantically doesn't work. I personally find it largely easier to read a text block wrapped at a fixed column size (say 80 char) than randomly at each sentence's end.

• On the "reading from source", adding line breaks after sentences directly goes against the very thing that makes languages readable. There's already a construct to separate thoughts, and it's paragraphs.

  The function of a paragraph is to mark a pause, setting the paragraph apart from what precedes it. -- https://en.wikipedia.org/wiki/Paragraph

  If two lines are following each other without a break, this is because they are part of a "unit of thought" or idea that goes together. This whole idea of semantic line breaks just goes against that. I'm fine with rules like breaking down long sentences into shorter ones etc., which are helpful rules for writing easy-to-read texts (it's even proven), but putting each sentence on a separate line is not that.

• Last but not least, I bet there's a very negative effect on reading speed, as your eyes are forced to scan back to the beginning of the next line at each sentence, and sentences are generally of quite varying lengths. Paragraphs in typography have attributes like indentation and spacing that guide the eye to the next paragraph. Here the proposed semantic line break throws all of that away.

Now, I would accept one argument, which is mentioned nowhere and is unrelated to semantic: line breaks might actually help in the context of version control, since VCSes like git are line-based, so adding a line break influences diffs and merges. But I'm not sure what's the extent of that influence, so I'd be wary to just add line breaks randomly.

[djeedai]

Updated with @Nilirad's suggestions. Not sure what's the usual process for comments, should I leave open for you to confirm resolution, or should I mark Resolved myself when I applied some change?

[Nilirad]

@djeedai, About semantic line breaks, I think you are mistaking how most markup languages are rendered. For most flavors of Markdown (GitHub comments and Discord excluded), a single line feed character (LF) does not result in a line feed or a new paragraph in the rendered HTML. To create a new paragraph you have to add at least two consecutive LF.

So, semantic line breaks do not affect the final output. This can be used at our advantage to improve the editing experience. For example, see the “Motivation” paragraph I reviewed a couple of days ago:

We need a data model for the Bevy Editor. We want to make it decoupled from the runtime data model that the Game uses, which is based on compiled Rust types (`World`, _etc._) with a defined native layout from a fixed Rust compiler version (as most types don't use `#[repr(C)]`), so we can handle data migration across Rust compiler versions and therefore across both Editor and Game versions. We also want to make it _transportable_, that is serializable with the intent to transport it over any IPC mechanism, so that the Editor process can dialog with the Game process to enable real-time Game editing ("Play Mode").

This is a single line of Markdown code. Since both VCS versioning and review comments are line-based, it is impossible to locate the thought I want to address without using quotes. It also forced me to stuff all the comments inside one single discussion. Giving each thought its own line instead will improve the reviewing experience:

We need a data model for the Bevy Editor.
We want to make it decoupled from the runtime data model that the Game uses,
which is based on compiled Rust types (`World`, _etc._)
with a defined native layout from a fixed Rust compiler version
(as most types don't use `#[repr(C)]`),
so we can handle data migration across Rust compiler versions
and therefore across both Editor and Game versions.
We also want to make it _transportable_,
that is serializable with the intent to transport it over any IPC mechanism,
so that the Editor process can dialog with the Game process
to enable real-time Game editing ("Play Mode").

The paragraph has been split into their different logical blocks, separating the cause from the effect, the premise from the consequence. In this way, each single thought can be referenced independently. As explained before, this does not change the HTML output, but it will improve writing (the physical structure of the text reflects its logical structure) and editing (most text edits are limited to a single thought, e.g. changing a wrong assumption about a logical consequence to the premise in the previous line).

For example, here I can already spot an issue with the paragraph: the speech does not flow well, because many thoughts are stuffed inside a single sentence, 5 in one case (not counting the parentheses), where a well constructed sentence is way shorter, with 3 or 4 thoughts at most.

EDIT: Re-reading your previous message, i think the word “thought” I used here is not completely appropriate. See them more like “logical blocks” of a sentence.

Not sure what's the usual process for comments, should I leave open for you to confirm resolution, or should I mark Resolved myself when I applied some change?

I think it depends. When a suggestion can be univocally applied, there is no reason to request feedback on that. In the other cases it is always better to wait for feedback.

[viridia]

So, before I begin, some context: I've written many game editors in my career, going all the way back to 1986. This includes editors I wrote while at Electronic Arts and Maxis. My most recent game engine / editor consists of 70k lines of TypeScript. So while my knowledge may be a bit out of date, I have a bit of experience in this realm.

The requirements given here have a number of shortcomings from my perspective.

The most important of these is the need to support custom types that are not just aggregates of standard types. In my current editor, there are about a dozen built-in editable types (float, int, bool, struct, vector3, etc.), but there are 24 custom field editors, that is, field editors that are specific to the game. One such example is the "transition predicate" type, which allows the artist to specify the conditions under which a quest transitions from one stage to another. Many custom types are context-specific, so for example while editing a dialogue tree, the "next state" field shows a dropdown list of the available states within the currently edited dialogue - not a global list of all possible states. Trying to edit this as a plain string would be an artist nightmare.

Also, many edit fields are closely integrated with the 3D view. For example, when editing a waypoint location field, a gizmo appears in the 3D window showing the location precisely. Trying to edit a location by typing in the coordinates is nearly impossible to get right.

Another function of the editor is to display important game data that is normally invisible - physics colliders, navigation meshes, waypoints, particle emitters and many others I could name. The artist needs to be able to enable and disable these overlays during the authoring process.

My current editor also supports a lot of game-specific editing modes, such as terrain sculpting, tree painting and biome painting. These are all plugins that are part of the general editing framework - the sculpting tools get mouse events just like any other ui widget, and handle the 3d picking via a set of common library methods.

On the other hand, some of the requirements listed here are non-goals from my perspective. I've never seen a case where an artist needed to update the game code in the middle of an editing session. Most of the editors I've worked with were a single binary that included both the game and the editor, and a typical artist workflow would pull a new version on a daily basis at most.

[nicopap]

I think this RFC is chewing a bit more than it can. I have more questions after reading the RFC than before.

• It seems to try to invent the concept of dynamic query? With the GameType component…
• But also an offset-based ReflectPath implementation?
• And a serialization format?
• And what's this story about "transactional access to the game state"? How does it relate to the "motivation" section?
• You talk about using the game state to "populate a User Interface". Re-reading the "motivation" section, I feel like UI shouldn't be a concern.
• Concerning implementation: I don't understand what "Names" do.
• You talk about a marker component to identify the "source" of a state. But I thought the state was stored in a World distinct from the actual ECS. In any case, what are you trying to distinguish between? If the editor and game processes are separate, no need to mark components.

Otherwise it's fairly unclear what it is trying to accomplish. For one, the language is very formal, weirdly lawyerly, and difficult to read. It reads like the Facebook Terms of Service. No ones wants to read a ToS! Here is a very useful resource on writing: https://www.plainlanguage.gov/resources/articles/dash-writing-tips/. You should follow those instructions. eg: you should avoid rare words like "forfeiting", and use less often the passive voice.

Also the RFC should probably be trimmed. It talks about a lot of different things, but I don't understand how those things relate to each others.

[djeedai]

The Motivation section has been rewritten to account for reviews. I will follow up on other sections too, to remove some tangential details and focus on the core purpose.