Design patterns for modern C++ (a series?)

[lefticus]

Books to reference:

• Head First Design Patterns (Java)
• Design Patterns (Classic Gang-of-Four / Pre-ISO C++)
• Hands on Design Patterns with C++
• Design Patterns in Modern C++20
• Modern C++ Design

[toRatnesh]

It will be much helpful if you try to teach these concepts with a project or select an open source project and then explain uses of these concepts in that project.

[fcolecumberri]

Maybe could be more appropriate for the [fill the blank].

[lefticus]

Maybe could be more appropriate for the [fill the blank].

In many ways I agree - series don't tend to do very well on C++ Weekly. But I've been trying to not make [fill in the blank] too much C++.

One huge advantage to [fill in the blank] is that I don't have a specific release schedule to maintain too.

[lefticus]

I have decided to go ahead and move this to [fill in the blank] so I can take more time developing it, and not have to fit it into the C++Weekly schedule.

[lefticus]

Reference based text book implementation of the observer pattern: https://compiler-explorer.com/z/MTdf55Y43

[lefticus]

Self registration with enable_shared_from_this cannot work https://compiler-explorer.com/z/sPbse6Yxs

[lefticus]

Separate registration with shared_ptr https://compiler-explorer.com/z/z51eW653c

[lefticus]

Auto registration and deregistration, with inheritance: https://compiler-explorer.com/z/qsjvh5WsT

[lefticus]

Generic reusable templated version: https://compiler-explorer.com/z/z5z9e9hEv

Do we need to handle the case where the provider can be destroyed before the observer? I don't think this is a normal use case to warrant the extra code necessary, but it can be implemented.

[lefticus]

A completely decoupled function based approach to observer: https://compiler-explorer.com/z/WGvo5rebE

[lefticus]

Self managing decoupled observer: https://compiler-explorer.com/z/dM8hnKs4z

[lefticus]

Slightly cleaned up version of the above: https://compiler-explorer.com/z/588cMveTc

[lefticus]

0 overhead Compile time registration of observers: https://compiler-explorer.com/z/7xqjs99sY

[lefticus]

Observations for Observer Pattern:

• It's close to signal/slots from Qt
• The classic examples assume a reference counted language
• The classic examples casually ignore the possibility of cyclic references by letting observers self-register with providers
• The classic examples assume objects that are not copyable or moveable
• The naive C++ reference-based implementation can easily lead to dangling references
• The naive C++ shared_ptr implementation can easily lead to cyclic references (see above)
• It's probably best to not allow observers to self register with providers - this decouples more of the code and reduces one path for dangling references and cycles
• It is possible to create a smart self-unregistering reference-based version in C++ that seems to be at least mostly safe
• If we force all registrations to happen at time of construction of the provider we can have a 0-cost version that's very hard to use wrong

[fcolecumberri]

Check this, instead of an “Observer” I created an “Observed”, I thinks you will like some of the stuff.

https://compiler-explorer.com/z/bGoEPT76x

[lefticus]

Check this, instead of an “Observer” I created an “Observed”, I thinks you will like some of the stuff.

https://compiler-explorer.com/z/bGoEPT76x

To say that "Lefticus hates std::function because it ads 3 extra asm instructions overhead" is a gross mischaracterization of my point. std::function adds a dynamic allocation and a cache miss for every function call, unless you are very lucky and the compiler can see all the way through your code and/or you have a "small" function.

It also has on the order of 10x the compile-time overhead of a lambda.

I don't say never use std::function, I say avoid it when you can.

Your code is the exact way that I say std::function should be used, and how I use it in my own examples above.

[fcolecumberri]

That part was meant to be just a joke, don't take it too seriously... I should have put an emoji there 🙃 my bad.

[lefticus]

Sorry, I'm taking the internet too seriously lately.

[oschonrock]

reading / parsing your versions with interest...

2 versions of my own, using the same practical example, which you may find interesting

https://godbolt.org/z/xo7PEaaj6

• both using weak_ptr for lifetime mgmt. When notifying and weak_ptr fails to lock, remove the slot.
• One using classic interfaces, virtual inheritance, polymorphism + dynamic cast
• The other using custom "slot functor" (essentially a lamba with captured weak_ptr, but providing external access to the weak_ptr). No virtual inheritance. Non-virtual inheritance mixin for common members. I did have std::functions, but reduced it to member function pointer as it didn't need the type erasure (incl heap allocation), because I have the captured observer instance separately already.

I agree that lifetime protection only needs to work one way, ie if Producer dies, all the slots die.

My mixin approach in the second one, is not quite as decoupled as Boost::Signals2 or your decoupled version. I probably like yours and Boost's better.

[lefticus]

Observer Pattern (aka publish-subscribe, aka Dependents)

• Head First Design Patterns: Chapter 2
• Design Patterns: Chapter 5
• Hands On Design Patterns: N/A
• Design Patterns in Modern C++20: Chapter 20
• Modern C++ Design: Mentioned but not discussed

TL;DR

Provides a method for a data provider to notify a data receiver when data base been updated.

In the C++ world these are generally called "Signal" and "Slot". Libraries already exist for this:

• https://github.com/larspensjo/SimpleSignal
• https://github.com/libsigcplusplus/libsigcplusplus
• https://www.boost.org/doc/libs/1_80_0/doc/html/signals2.html (predates C++11)
• http://sigslot.sourceforge.net/
• https://github.com/palacaze/sigslot
• Qt's signals/slots

Synopsis

• Two parts
  • Data Provider / Subject
  • Data Observer

Some mechanism exists to tie the Provider and the Observer together.

Changes are notified either:

• Push (Provider informs Observer of change)
• Pull (Observer requests lasts data from Provider)

The provider must decide how and when to update its publicly available state.

Key Details

• State must be updated before notifications are sent. Some automated handling for this is best
• Care must be taken to avoid dangling references
• If the Provider and Observer are both aware of each other, then care must be taken to avoid cycles (in reference counted scenarios)

Design Considerations

• automatic registration / deregistration of connections to match object lifetimes?
• shared_ptr based reference counted objects to avoid problems with object lifetime?
• how much coupling of subject and observer are you willing to tolerate?
• is thread safety a concern?
• should change->notification be synchronous or asynchronous?
• should the signal keep a copy / own the last known value?
• Is the relationship 1:N, 1:1, N:1, N:N?
• If you have 1:1, you can use an intermediary to create all of the other options

Everything from

• compile-time fixed connections

to

• multithreaded message queues

Are possible.

Classic OOP Model (Head First Design Patterns / Design Patterns)

ISubject, IObserver and concrete implementations of the interfaces. ISubject::Attach attaches an observer to be notified.

These are very intrusive, requiring heavy lifting and inheritance / implementation by the Subject and Observer.

• Naive reference-based C++ implementation has easy to invoke object lifetime issues: https://compiler-explorer.com/z/MTdf55Y43
• Self-registering shared_ptr based version (Head First Design Patterns) - does not work well because shared_from_this does not work during object construction: https://compiler-explorer.com/z/sPbse6Yxs
• Non-self-registering shared_ptr based implementation - works - https://compiler-explorer.com/z/z51eW653c
• Self registering and unregistering & based implementation - https://compiler-explorer.com/z/qsjvh5WsT
• Templated and reusable version of classic design with self-registration and unregistration: https://compiler-explorer.com/z/z5z9e9hEv

Decoupled Function Based Observer

In these versions we make a reusable component that does not require inheritance, but have a slightly more manual method of registration:

• https://compiler-explorer.com/z/WGvo5rebE
• Completely self-managing registration version: https://compiler-explorer.com/z/588cMveTc

constexpr Friendly / very light weight versions

• 0 overhead compile-time based registration: https://compiler-explorer.com/z/7xqjs99sY
• connection object based implementation: https://compiler-explorer.com/z/va76j3WGs

[oschonrock]

couple of thoughts..

I like your registerDestructor callbacks (compared to my shared_ptr / weak_ptr), because they act instantly (rather than on notify) and don't use a "heavy" shared_ptr / weak_ptr / mutex.

However, it depends what we are trying protect against:

• For an embedded target with single thread of execution your solution is probably better (shared_ptr is overkill), as it protects against simple usage mistakes.

• For a larger, multi-threaded target (perhaps notify happens async over network), your solution may not provide full thread safety, without adding more complexity?

interestingly boost::signals2 gives the user options for what kind of lifetime protection they want: "Nothing" or "shared_ptr" style.

Such customisation points are probably beyond what you could reasonably cover?

[lefticus]

couple of thoughts..

I like your registerDestructor callbacks (compared to my shared_ptr / weak_ptr), because they act instantly (rather than on notify) and don't use a "heavy" shared_ptr / weak_ptr / mutex.

However, it depends what we are trying protect against:

* For an embedded target with single thread of execution your solution is probably better (shared_ptr is overkill), as it protects against simple usage mistakes.

* For a larger, multi-threaded target (perhaps notify happens async over network), your solution may not provide full thread safety, without adding more complexity?

interestingly boost::signals2 gives the user options for what kind of lifetime protection they want: "Nothing" or "shared_ptr" style.

Such customisation points are probably beyond what you could reasonably cover?

I think I've addressed your comments in my uber "Observer Pattern" comment. I clearly cannot address all design considerations and options in one episode. Honestly I think this could be a 2 day class. We could touch on everything from constexpr and tmp to thread safety.

[oschonrock]

Yeah there is a lot of potential depth here...

It's a really interesting style of topic though, even if everything cannot not be ultimately solved to the nth degree, within scope.

[lefticus]

Relationship diagram: https://www.researchgate.net/figure/Design-patterns-relationships-classification_fig2_224930182

[lefticus]

Coming in Ep373

[W4RH4WK]

There is an excellent article comparing different implementations of signal/slots C++ libraries:

https://julienjorge.medium.com/testing-c-signal-slot-libraries-1994eb120826

[NikBomb]

Hi Jason,

A great episode. If I can make a suggestion I would add to the list of books the one from Klaus:

C++ Software Design: Design Principles and Patterns for High-Quality Software

[bert-laverman]

Nice, by starting with/focusing on the Observer pattern, you kind of made a start with #225. What I liked about your episode is how you highlighted the exact issue (challenge?) that modern C++ brings for people like me, who have been doing mostly Java (or C#) and want to do C++ "on the side." The thing is, for me modern C++ feels like a language that tries to support as many as possible of modern design patterns while continuing to cater to those who distrust the compiler's ability to "make the correct decision." The absolute majority of large software systems perform acceptably without being "Stack First." C++ never accepted that premise and as a result library builders are constantly forced to provide multiple versions of their code, depending on the percentage of data that can be managed on the heap.

Your episode kind of suggests that the question should be based on the objects' lifecycle, whereas I would put performance first. Nevertheless, the result of always having to ask that question leads to a lack of higher-level libraries, because of the need to cater to all permutations of on-stack vs on-heap of the different supported types. A case in point here is the Reactive libraries available, which allow you to choose if you want your objects on heap, at the cost of an increased complexity and maintenance cost for that library.

Using the heap frees you from a lot of the puzzles and limitations that C++ can trip you up with, but you must be able to afford it. If I pass an instance of Child to a function or method expecting a Parent, I expect to be able to test for the type and (if needed) cast it. In C++ this is only supported on the heap, which makes the heap a default choice for many cases, leading to ridicule by those decrying the associated costs. Look also at the ridiculous interface around exceptions, with std::exception_ptr and std::current_exception(). Sure I can build something generic to pass that pointer to a method, but if you want to get the actual exception object, you have to jump through a number of hoops. You can only rethrow it and then use catch clauses to find out what the type of the object was, but that's about it, because it may have been on-stack or even already destructed. Exception handling in the context of lambdas becomes a nightmare.

[LB--]

@bert-laverman A couple of points you raised don't appear correct to me:

If I pass an instance of Child to a function or method expecting a Parent, I expect to be able to test for the type and (if needed) cast it. In C++ this is only supported on the heap, which makes the heap a default choice for many cases

This is not true, polymorphism and virtual functions and runtime type information all work fine regardless of where the memory is allocated from. It's just that many people find it convenient to mix different derived types in a container of base pointers, but this isn't the only way to do things. You could for example have fixed stack arrays for each derived type and then just go through each one and pass them to the general-purpose functions that take pointers or references to base, and everything still works the same as before.

You can only rethrow it and then use catch clauses to find out what the type of the object was, but that's about it, because it may have been on-stack or even already destructed.

This is also not true, the throw expression copies or moves the provided exception object into the thread's exception storage, so the 'original' is always destroyed by the time stack unwinding begins. This does not slice or damage the object type in any way. If you need to re-throw an exception from a catch clause you would use throw; without arguments so that it re-uses the existing object instead of copying or moving which could slice. But for the initial throw it is not possible for slicing to occur if you create the exception object in the parameter of the throw expression. The only way to encounter slicing is if you try to throw a base reference to a more-derived type, or if you catch by value instead of by reference, both of which are very strange and unlikely to be encountered in good code.

Also, I would like to know, what would you even do with the information about the type of the exception? Almost always by the time you want to inspect an exception_ptr you would want to do so for the purposes of catching the exception as usual anyway, so there is no reason to inspect the type in advance. If you want an interface more like std::any then you could just throw std::any as your only exception type.

[bert-laverman]

@lefticus

This is not true, polymorphism and virtual functions and runtime type information all work fine regardless of where the memory is allocated from. It's just that many people find it convenient to mix different derived types in a container of base pointers, but this isn't the only way to do things. You could for example have fixed stack arrays for each derived type and then just go through each one and pass them to the general-purpose functions that take pointers or references to base, and everything still works the same as before. Ok, I am caught here by my bad wording of what I intended to say. Perhaps I should have added some actual code:

class Parent {
... // Details left out
}

class Child : public Parent {
... // details left out
}

void hiThere(Parent p) {
  // I now have nothing to test or cast to, whatever the object was, it is now a Parent
}
void hiAgain(Parent& p) {
  // Here I have a potentially testable object, as long as it is alive.
}

What I should have said is that objects on the stack are dangerous to use as a referenced or (Bog forbid) pointed-to parameter, because of the potential for it being destructed by the time it is used. On the heap, my only problem is that I need to ensure it does not become untraceable garbage. That is such a reduction in complexity, that objects on the heap become a preferable solution.

As for the exception, that is definitely my bad, I have probably wrongly interpreted the reasoning in a piece of text that advocated to always catch by reference.

As to why I want this: In CsSimConnect I can have a generic GetData method that looks up the type of the actual type parameter to see which simulator values need to be placed in what fields of the object returned, optionally even doing some conversions. For new types (ones that haven't been encountered before) it uses reflection to gather the fields, checking for an annotation (called an attribute in C#) that provides the simulator linkage information. This allows me to have the code using the library to be very clean, with references to simulation variables neatly described next to the fields that will contain the actual data retrieved. Without it, and this is how it works in the original SimConnect SDK, I have to use explicit calls for each field to tell the library which simulation variables I want, and copy each field manually from a block of data returned to me every time the simulator sends over a fresh set.

Admittedly the SimConnect SDK is archaic, not having had useful updates since the Windows XP era beyond paying some lip service to the existence of modern C, so the comparison becomes one of extremes, but now I am able to do:

public class AircraftData
{
    [DataDefinition("ATC TYPE", Type = DataType.String256)]
    public string Type { get; set; }
 
    [DataDefinition("ATC MODEL", Type = DataType.String256)]
    public string Model { get; set; }
 
    [DataDefinition("ATC ID", Type = DataType.String256)]
    public string Id { get; set; }
 
    [DataDefinition("ATC AIRLINE", Type = DataType.String256)]
    public string Airline { get; set; }
 
    [DataDefinition("ATC FLIGHT NUMBER", Type = DataType.String256)]
    public string FlightNumber { get; set; }
 
    [DataDefinition("TITLE", Type = DataType.String256)]
    public string Title { get; set; }
 
    [DataDefinition("NUMBER OF ENGINES", Units = "Number", Type = DataType.Int32)]
    public int NumberOfEngines { get; set; }
 
    [DataDefinition("ENGINE TYPE", Units = "Number", Type = DataType.Int32)]
    public int EngineType { get; set; }
}

...
  // Get info about the currently selected aircraft
  AircraftData data = DataManager.Instance.RequestData<AircraftData>().Get();
...
  // Set up status-bar updates
  DataManager.Instance.RequestData<AircraftData>(ObjectDataPeriod.PerSecond, onlyWhenChanged=true)
                    .Subscribe((AircraftData data) => this.Dispatcher.Invoke(UpdateStatusBar(data)));

Since, for several reasons, even absolute beginners want to use C or C++ for their add-ons, I am working on a C++ version of this library.

Cheers, Bert

PS Yes, this means there are actually add-on developers that start learning C when they build their first MSFS add-on. Asobo does not prioritize the SimConnect SDK much unless a major add-on developer manages to tickle them (or Microsoft) enough. Many of them have been around since before FS-X (Windows XP era) and the cost of switching languages is high, especially if the SDK expects C.