Improve guidance on evolving message contracts

[kbaley]

There are a lot of intricacies involved in evolving message contracts and our current guidance can be improved. Some examples:

• Outline pros and cons of inheritance vs. brand new contracts (with a version postfix in the name) vs. updating existing contracts
• Include information about versioning messages by namespace
• Clearer descriptions on the effects of upgrading subscribers/receivers first vs. publishers/senders
• More code samples showing explicitly what happens when properties are added/removed
• Specific guidance for evolving saga classes
• Effects of different serializers (e.g. how they handle missing properties)
• Methods for determining the version of a contract if necessary
• Suggest unobtrusive mode more strongly in the "sharing contracts" guidance

This could work well as a tutorial spread over multiple pages where users can download the code at different stages in a contract's evolution.

[timbussmann]

version that was suggested during #5622 - click to expand

In message-based systems, the messages are part of a contract, which defines how services communicate.

Evolving contracts over time is challenging, and an appropriate strategy should be reviewed and customized for each system. When evolving message contracts, consider the following:

• Endpoints updated to the latest message contract might still receive messages using the old contract. Senders might still use the old contract, or not all in-flight messages (messages waiting to be consumed in input queues) have been processed before the upgrade.

• Endpoints updated to the latest message contract might send messages, using the new contract, to endpoints still based on the old contract version.

Generally, the problem can't be resolved at the infrastructure level; therefore, NServiceBus users must analyze their systems, consider how they are expected to evolve, and define the strategy which will make the most sense in their particular circumstances.

This article presents basic guidelines for choosing a contract evolution strategy, avoiding common mistakes, and ensuring that contracts will be easy to evolve over time.

Note: Ensure that message contracts follow the general messages design guidelines.

Techniques

There are different techniques to evolve message contracts, each one with advantages and disadvantages. When selecting a technique it's important to carefully plan the migration strategy, the needs of receivers/subscribers and senders/publishers.

New message type

Ship multiple versions of a message contract in the same assembly by creating a new message type for the new contract version. The new type typically indicates its relationship via its name, e.g.

// first version of the message contract
public class CreateOrder : ICommand
{
    ...
}

// new version of the message contract
public class CreateOrderV2 : Icommand{
    ...
}

• Consumers can be updated to the new version independently from updating the contract assembly, giving more flexibility in planning the upgrade process.
• Allows more flexibility about the contract changes as no type-level compatibility must be ensured.

When updating commands/messages, make sure receivers have a handler for both types. This ensures they can process in-flight messages or messages sent by endpoints still using the old contract.

When updating events, update all subscribers to have handlers, and to subscribe, to both events before changing publishers.

Use inheritance

Ship multiple versions of a message contract in the same assembly by using inheritance. The new contract version inherits from the previous version, gaining all its parent's properties.

// first version of the message contract
public class OrderCreatedEvent : IEvent {
    ...
}

public class OrderCreatedEventV2 : OrderCreatedEvent {
    ...
}

• Can only be used to make additive changes
• Flattening the inheritance chain / removing older versions is more difficult compared to the other options
• Something about multiple/all message handlers being invoked
• Something about routing complexity?

When updating commands/messages, update all senders to the new version before adding new message handlers for the new version to receivers. This might significantly delay the time until the new message contract can be implemented by receivers.

When updating events, update publishers first to publish only the latest version of the event. Subscribers using the old message contract will also receive the event but they will process the messages as if they were the previous version.

Adding data to existing contracts

When modifying an existing contract, contract consumers updating to the latest contract version will only know this version of the contract. When only making additive changes, the serialization behavior can be used to ensure backwards compatibility with older versions of the contract.

// first version of the message contract
public class CreateOrder : ICommand {
    ...
    // this property has been added in the latest version:
    public int? SomeProperty { get; set; }
}

• Endpoints using the old version of the contract can process newer versions since the additional data is just being ignored
• The new data should be using nullable types to ensure endpoints can distinguish between missing data and intentionally set default values (e.g., 0 when using int).
• Endpoints using the new version of the contract need to expect the new data to be missing if there are still messages in the queue when updating, or if endpoints can send/publish using the old contract.

When updating commands/messages, the receiver of the message should be able to handle messages that have the new property missing. If that is not possible, all senders must be updated first, potentially delaying the time until the new data can be fully used by the receiver.

When updating events, update publishers first. Subscribers using the old contract will simply ignore the additional data and can be updated one at a time.

Removing data from existing contracts

When modifying an existing contract, contract consumers updating to the latest contract version will only know this version of the contract. When removing properties from a contract, extra care has to be taken since this can impact endpoints still referencing the old version of the contract that aren't able to handle the missing data.

When updating commands/messages, update receivers first to ensure they are no longer depending on the property being removed. During this time, senders might still send the additional data till they are updated.

When updating events, update subscribers first to ensure they are no longer depending on the property being removed. During this time, publishers still need to provide the data being removed.

Note: Renaming properties on a contract is equivalent to removing one property and adding another property.