Should class member memory layout always be in-order?

[BlakeTheAwesome]

Background

I've just been reading through through the design document and saw this line in the Classes section:

The order of the field declarations determines the fields' memory-layout order.

While this is true of C++ code, I'm unconvinced that it is a good default behaviour as it means programmers should always consider member packing/padding when creating their interfaces as opposed to optimising for readability. Specifically it means members should generally be grouped based on their data types, and not on logical groups of what is used together.

For example, consider these two classes:

class LogicallyGrouped {
  var titleText: String;
  var titleBold: bool;

  var subTitleText: String;
  var subTitleBold: bool;
};

class DataSizeGrouped {
  var titleText: String;
  var subTitleText: String;

  var titleBold: bool;
  var subTitleBold: bool;
};

LogicallyGrouped will likely be easier for programmers to read, think about and expand upon, but it has a bunch of extra padding bytes between titleBold and subTitleText. DataSizeGrouped is the 'better' way to write the class, but for machine-related reasons not human-related reasons.

A benefit of the current ordering is that it is explicit and therefore has better interoperability with C and C++, as well as potential uses for serialising data by copying memory directly. For this reason I believe we have to include a way order members, but I do not believe this should be the default case, generally preferring to include C/C++ headers directly when accessing those structs, and utilising some form of reflection for serialisation.

A related issue is struct packing, as there are times when the padding bytes between members (or at the end of a struct) should be omitted entirely for compactness during transportation. While this is a rarer case, I believe it is worth considering at the same time so a clean consistent solution can be reached for all memory layout situations - similar to how C# uses its StructLayoutAttribute to define the layout type and packing.

Proposal?

By default the memory-layout of structs should be automatically defined by the compiler, with the ability for users to opt-in to a strict layout. I'm unsure about the best way to annotate a class, so for now I'll suggest keywords to spark discussion.

All classes should have a memory_layout that can be optionally defined, and set to one of four values:

• MemoryLayout.Fastest (default): The compiler decides what ordering makes the most sense. This is likely to be the same as Smallest to begin with, but it is open to platform-specific or profile-guided optimisations should people come up with them.
• MemoryLayout.Smallest: The compiler orders the members based on their sizes to minimise padding bytes. This is currently what programmers should write by default in a language such as C++.
• MemoryLayout.Sequential: The compiler orders the members based on the order of their declarations. This is the current ordering, and should be automatically added to #included C++ classes.
• MemoryLayout.Explicit: The programmer should add a layout_order which is an array of field names specifying the order they want the members in. This is separate from Sequential as it allows the programmer to optimise their class declaration for human-reading, while still allowing interoperability with classes defined elsewhere.

All classes should have a memory_packing which can be set to an integer to specify packing similar to MSVC's #pragma pack(x). If it is not set it uses the current default packing.

Example:

class LogicallyGroupedButActuallySequential {
  var titleText: String;
  var titleBold: bool;

  var subTitleText: String;
  var subTitleBold: bool;

  memory_order: MemoryLayout.Sequential;
};

class LogicallyGroupedButCustomOrdering {
  var w: f32;
  var x: f32;
  var y: f32;
  var z: f32;

  memory_order: MemoryLayout.Explicit;
  layout_order: ["x", "y", "z", "w"];
};

class ShrunkForTransport {
  var hasData: bool;
  var floatData: f32;

  memory_order: MemoryLayout.Sequential;
  memory_packing: 1;
};

Some extra considerations:

• Should memory_packing affect padding between members as well as trailing padding, or is there a reason to handle these situations separately?
• Should layout_order allow for complex expressions and not just an array literal? There may be use cases for conditionally-defined orderings - for example if included external code rearranged their fields between versions and we want to keep all that logic in one place:

  class LogicallyGroupedButCustomOrdering {
    var w: f32;
    var x: f32;
    var y: f32;
    var z: f32;
  
    layout_order: if UsingV1 then ["w", "x", "y", "z"] else ["x", "y", "z", "w"];
  };
  

[jwtowner]

I think once Carbon gets metaprogramming and static reflection defined and implemented, we'll have a lot more power over how memory order can be transformed from the default order. Same with C++ if it ever gets metaclasses as part of the core standard. One litmus test for Carbon's metaprogramming facilities would be to have it automatically generate at compile time a data-oriented, structure-of-arrays order system class as well as object-oriented views from a single interface or archetype acting as a model. Something similar to the idea of Protocol Buffers or FlatBuffers.

So maybe some of this could be implemented as a library on top of the core metaprogramming language features, at least insofar as for things like serialization, transport, data-transfer objects, interop, ABI compatibility, etc.?

[BlakeTheAwesome]

I think once Carbon gets metaprogramming and static reflection defined and implemented, we'll have a lot more power over how memory order can be transformed from the default order.

I haven't read through the metaprogramming proposal yet so it may well make more sense than the keyword-based stuff I suggested. What I would like discussed (maybe once metaprogamming is fully defined) is what the default ordering should be for the classes - because I definitely think the default behaviour should be that you don't have to think about it to get a good result.

It always feels odd needing to teach junior C++ programmers about memory layouts before any of their classes can be used when 99% of the time it shouldn't matter. We should only need to be explicit about it in the rare cases when it's actually required, and by being explicit at those times it also puts up a warning signal to anyone modifying the class that the specific order matters and they can't just swap two bools around to make it read nicer.

While this can be added in a later language version by adding tooling to make all existing classes Sequential when upgrading to a version with Fastest-by-default, it'd be nice if early code could have good defaults.

[OlaFosheimGrostad]

While this is true of C++ code, I'm unconvinced that it is a good default behaviour as it means programmers should always consider member packing/padding when creating their interfaces as opposed to optimising for readability.

Not only that, but performance is cache-line sensitive which suggests that layout optimizations should be based on profiling and the specifics of the target hardware.

So yes, I agree, the compiler should be able to choose the layout unless the programmer specifies otherwise.

[jwtowner]

It always feels odd needing to teach junior C++ programmers about memory layouts before any of their classes can be used when 99% of the time it shouldn't matter.

Definitely true.

[geoffromer]

...it means programmers should always consider member packing/padding when creating their interfaces as opposed to optimising for readability.

As you observed later in the thread, member packing/padding almost never matters. Why should programmers always consider it, rather than waiting for performance measurements to tell them it needs to be considered?

[chandlerc]

Another factor here is that we say in our goals:

Code should perform predictably.

Having an implementation-dependent order of fields will create significant variability in performance. And perhaps even more important, the layout won't be visible and transparent to the developer.

If you have a large class with many fields, but the first two are i32s and very hot while the rest is cold, but the compiler helpfully locates those two fields far apart in padding bytes with other fields, it would be I think very hard for a reader to understand why code accessing just those fields has unexpectedly poor performance.

This is especially true for developers with a background in C++ (or C or many other languages) where this property is very fundamental and expected.

The fact that layout rarely matters isn't an especially powerful motivation here because "the order they are declared in" is a perfectly fine choice when it doesn't matter.

When it does matter, having the order be directly and visibly expressed, and in the same familiar manner as most other languages in this area seems like the right choice to me.

(Also marking this as an issue for leads to get a decision here.)

[chandlerc]

The leads went back over this and remain happy with member memory layout being in-order for now. You can see some of the rationale in my comment above: https://github.com/carbon-language/carbon-lang/issues/1680#issuecomment-1200079811

We would also expect to see approaches like tools that suggest source edits to optimize memory or metaprogramming facilities to synthesize a particular memory layout programmatically in the future, but for classes written in-order, we expect them to be in memory in that order (when observable).