Shared installation support

[magreenblatt]

Overview

This issue tracks the implementation of shared installation support for CEF. Use of a shared installation will be optional and will likely come with restrictions on available APIs and/or installation behaviors. Embedders that require non-standard binaries (like proprietary codecs enabled or custom CEF/Chromium modifications) will continue using a bundled installation.

A shared installation will address these primary concerns:

1. Installation size. Applications that embed CEF must currently bundle all CEF binaries with their application. This can lead to large installer sizes (> 100MB), large installation sizes (> 300MB), and multiple installations of CEF on a single device.
2. Complexity. On MacOS, embedders must structure their applications in certain ways that are not always ideal. On Windows/MacOS, embedders that enable the sandbox must link a static library that injects potentially undesireable build dependencies.
3. Security. CEF is based on Chromium which is constantly updated to address bugs and security issues. Apps running an older version of CEF/Chromium may have known vulnerabilities that put the user’s computer or data at risk. Moving to a shared CEF install could allow automatic updates of the CEF/Chromium version for all participating apps.

WebView2 Evergreen, an alternative to CEF, supports the concept of a system installation shared by multiple application installations. WebView2 is unfortunately Windows-only and Microsoft has abandoned plans to support other platforms. We can, however, use WebView2 as a reference model for implementing this functionality in CEF.

This implementation will involve the following phases:

Phase 1: Shared runtime behavior

BACKGROUND: CEF/Chromium currently hard-codes some assumptions about usage in a single application.

GOAL: Run multiple CEF-based applications with CEF located in a separate/shared directory. Each application will run independently and be completely isolated from the others in memory and on disk.

RELATED PROBLEMS: The default "User Data" directory is not application-specific, each platform has it's own behavioral quirks related to path discovery (#3749), loading behavior of libcef is not consistent across platforms (related).

Phase 2: Back/forward ABI compatibility

BACKGROUND: The CEF library/framework exports a C API that isolates the user from the CEF runtime and code base. The libcef_dll_wrapper project, which is distributed in source code form as part of the binary release, wraps this exported C API in a C++ API that is then linked into the client application. The code for this C/C++ API translation layer is automatically generated by the translator tool.

GOAL: Ensure that a pre-compiled CEF-based application can successfully run against multiple different major/minor milestone versions of CEF/Chromium (both older and newer versions; WebView2 example).

RELATED PROBLEMS: CEF version (API hash) checking is too strict, C structs and enums are not consistently versioned, new API surface is not added in a backwards-compatible manner, cef_sandbox.lib linking introduces build dependencies on Windows/MacOS.

See additional details below.

Phase 3: Shared installers & guidelines

BACKGROUND: After completion of Phases 1 and 2 it will be technically feasible to install CEF in a shared location on a user's device and utilize that shared install from multiple CEF-based applications.

GOAL: Provide official CEF shared installers with associated documentation and guidelines on usage for each platform.

RELATED PROBLEMS: Installers and installation behavior will have platform-specific requirements, CEF embedders need testing/validation support to "guarantee" compatibility between milestone versions, some form of "version pinning" will likely be required.

Explicit Non-Goals

We do not anticipate significant changes to CEF/Chromium behaviors or CEF C++ APIs or as part of this effort. In that vein, we have defined some explicit project non-goals:

1. Sharing the CEF runtime between apps in-memory (e.g. not duplicating the browser process). This has pretty substantial security and privacy implications and would require something “new” on top of CEF/Chromium to handle the multiplexing. See related WebView2 discussion here.
2. Supporting multiple versions or initializations of CEF in the same process. Applications that utilize CEF via a plugin-style architecture will still need to ensure a singleton loading and initialization of CEF.
3. Surfacing deprecated APIs in C++. Applications building against the CEF C++ API (libcef_dll_wrapper) will always choose a single API version (see below). Older/newer API versions may exist in the C API to the extent necessary for ABI back/forward compatibility, but this will be transparent to C++ API consumers.

[Hethsron]

@magreenblatt Good news

[magreenblatt]

Phase 2: Back/forward ABI compatibility

These are the proposed rules for CEF API design after Phase 2:

Enums:

• All enums get a _LAST value.
• New values are always added at the end (except in cases where the value is explicitly set).
• Old/deprecated values are renamed to _DEPRECATED (or _REMOVED) but not removed.

Structs:

• All structs get a size_t size member at the beginning.
• New members are always added at the end.
• Old/deprecated members are renamed to _deprecated (or _removed) but not removed.
• Existing members never change type (old may be renamed and new added with the same name).

Member methods:

• Return value and parameters never change for existing methods.
• New methods are always added at the end of the class. This is relevant for the C API representation of the class as a struct of function pointer members -- ordering may be applied in the translation layer (see below).
• Old/deprecated members in the C API struct are given an opaque placeholder pointer type (uintptr_t) to preserve binary compatibility (memory layout) with older versions.
• The existing method may be deprecated and a new method added if return value or parameters need to change. C++ naming may use polymorphism (e.g. multiple OnEvent with different params) but C API will likely use numbering (e.g. on_event as the original, on_event[2...N] or on_event_[VERSION] as the replacement). See examples below.

Static methods & functions:

• Return value and parameters never change for existing methods/functions.
• Ordering doesn't matter as these symbols are exported directly from the library/framework.
• The existing method/function may be deprecated and a new method/function added if return value or parameters need to change. C++ naming may use polymorphism (e.g. multiple CefDoWork with different params) but C API will likely use numbering (e.g. cef_do_work as the original, cef_do_work[2...N] or cef_do_work_[VERSION] as the replacement).

API versioning:

• Any API change (deprecated, changed or new) results in a new API version with associated #define (e.g. #if CEF_API_VERSION >= 13201). Exact format/source for API version #define values is still TBD (see below).
• Client decides what API version to use (up to the current version) at build/compile time (e.g. by adding CEF_API_VERSION=13201 to project defines). This has the following impact:
  • Client will report the selected API version to the CEF library/framework during initialization as a hint for internal logic (like enum/struct size expectations, correct client callback version, etc).
  • The application will only be compatible with CEF versions that implement this API version or newer. For example, a client can select the M132 API when building with an M132+ binary distribution. The client will then be compatible with all CEF versions at M132 or newer, but not with versions older than M132.
  • The client-side CEF C++ API (contents of include/ and libcef_dll_wrapper) will change based on the API version. Deprecated and newer APIs will be removed (compiled out) so the the CEF C++ API remains the same for that version even when using newer CEF binary distributions. Note that libcef_dll_wrapper may still need to be recompiled when moving to a new binary distribution as C++ technology can change (like moving to C++20); this does not impact CEF's ABI compatibility.
• The library-side CEF C++ will have access to all API versions of client callbacks (current and deprecated). It will use the reported/selected client API version to determine which callback version to execute. See details below.
• The library-side CEF C++ will implement all API versions of methods/functions (current and deprecated). Deprecated or new versions may error out based on the reported/selected client API version.
• Test coverage will be added for API version compatibility. For example:
  • ceftests will be built with different API version values to verify that compilation and tests still pass when specifying older API versions.
  • ceftests will be run with older/newer CEF binaries to verify that tests still pass at the same API version (within the version range that is expected to be compatible).
• Newer CEF versions may occasionally drop support for older API versions (see example below). This may occur on a schedule (e.g. annually), or as a result of incompatible Chromium changes breaking existing APIs in irresolvable ways. When this occurs it will be clearly communicated and supported by installers (from Phase 3) that can pin existing clients at older CEF versions.

[magreenblatt]

Exact format/source for API version #define values is still TBD.

The API version number should be incremental (to work with logical operators) and might be based on the CEF version where it was first introduced. For example, XXXZZ where:

• XXX is the Chromium major version (e.g. 132)
• ZZ is an incremental number (e.g. '01', '02', etc) that gets reset to 00 when the Chromium major version changes.

The CEF version number currently changes when the CEF C/C++ API changes, so this format would be easy to inspect ("Ah, the 13201 API was first introduced in CEF version 132.1!") while still allowing plenty of room for future API revisions (up to 99 in a single major version).

We can support this with a new tool that works as follows:

• version_manager.py --next: Outputs the next available (new) API version to be used in #defines in CEF's C/C++ headers. Note that this would be based on the current master/beta branch version since we don't often cherry-pick new/breaking API changes to stable branches. Updating the API version in a stable or older branch would be considered divergent (unsupported) behavior since it breaks backward/forward version compatibility (see alternative below).
• version_manager.py --apply: Computes the API hash for the new API version based on current C/C++ headers (with version #defines applied). Writes the new version + API hash to a JSON file that gets updated to the CEF repo in the same commit (replacing the existing cef_api_hash.h file usage).
• version_manager.py --check: Reads the JSON file and computes/compares the API hash for each supported version (applies version #defines) to verify that the API for older versions has not been changed accidentally.

The CEF translator tool does not currently support #defines in CEF C++ headers. Since we'll need to add support for version #defines we might also add support for platform defines (OS_WIN, etc) at the same time.

A few other related points:

• Apps using the shared installers should always explicitly specify the API version when building (e.g. by setting CEF_API_VERSION=13201 in their project config). Compatibility information (API version, optional min/max CEF version) will also need to be specified when running the installer.
• By default, when CEF_API_VERSION is not explicitly set, apps will build using the "current" (newest) API version available for that CEF branch.
• We may wish to support "experimental" APIs that are compiled out by default when CEF_API_VERSION is explicitly set. This would allow us to iterate on new APIs that are not (yet) subject to the backward/forward compatibility constraints.
• Documentation will always be generated using the "current" API version. This should be fine since documentation is already uploaded to version-specific directories (example).

[magreenblatt]

Here are some concrete examples of API versioning.

Original version

// In C++ API:

/*--cef(source=library)--*/
class CefTestObject : public CefBaseRefCounted {
 public:
  ///
  /// Create the test object.
  ///
  /*--cef()--*/
  static CefRefPtr<CefTestObject> Create();

  ///
  /// Set a value.
  ///
  /*--cef()--*/
  virtual void SetValue(int value) = 0;

  ///
  /// Returns true if a value was set.
  ///
  /*--cef()--*/
  virtual bool HasValue() = 0;
};


// In C API (auto-generated):

typedef struct _cef_test_object_t {
  ///
  /// Base structure.
  ///
  cef_base_ref_counted_t base;

  ///
  /// Set a value.
  ///
  void(CEF_CALLBACK* set_value)(struct _cef_test_object_t* self, int value);

  ///
  /// Returns true (1) if a value was set.
  ///
  int(CEF_CALLBACK* has_value)(struct _cef_test_object_t* self);
} cef_test_object_t;

Method parameter changed in version 13201

// In C++ API:

/*--cef(source=library)--*/
class CefTestObject : public CefBaseRefCounted {
 public:
  ///
  /// Create the test object.
  ///
  /*--cef()--*/
  static CefRefPtr<CefTestObject> Create();

  // NOTE: BUILDING_CEF_SHARED is defined when building libcef. It has access
  // to all versions but the client (which defines CEF_API_VERSION) does not.
  // We need to be careful not to change methods in ways that are unsupported
  // by polymorphism. For example, if the return type changes then we also need
  // to rename the C++ method.
#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION < 13201
  ///
  /// Set an integer value.
  ///
  // NOTE: CEF metadata specifies the supported version range.
  /*--cef(removed_version=13201)--*/
  virtual void SetValue(int value) = 0;
#endif

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION >= 13201
  ///
  /// Set a double value.
  ///
  // NOTE: Need to give the new method a different/unique name in the C API.
  /*--cef(added_version=13201,capi_name=set_value2)--*/
  virtual void SetValue(double value) = 0;
#endif

  ///
  /// Returns true if a value was set.
  ///
  /*--cef()--*/
  virtual bool HasValue() = 0;
};


// In C API (auto-generated):

typedef struct _cef_test_object_t {
  ///
  /// Base structure.
  ///
  cef_base_ref_counted_t base;

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION < 13201
  ///
  /// Set an integer value.
  ///
  void(CEF_CALLBACK* set_value)(struct _cef_test_object_t* self, int value);
#else
  // NOTE: Using an opaque pointer type to reserve space in the structure.
  // A client building at API version >= 13201 will never access this pointer.
  uintptr_t set_value_removed;
#endif

  ///
  /// Returns true (1) if a value was set.
  ///
  int(CEF_CALLBACK* has_value)(struct _cef_test_object_t* self);

  // NOTE: New function pointers are added at the end of the structure.
  // A client building at API version < 13201 will never see this pointer.
#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION >= 13201
  ///
  /// Set a double value.
  ///
  void(CEF_CALLBACK* set_value2)(struct _cef_test_object_t* self, double value);
#endif

} cef_test_object_t;

Method parameter changed again in version 13301

// In C++ API:

/*--cef(source=library)--*/
class CefTestObject : public CefBaseRefCounted {
 public:
  ///
  /// Create the test object.
  ///
  /*--cef()--*/
  static CefRefPtr<CefTestObject> Create();

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION < 13201
  ///
  /// Set an integer value.
  ///
  /*--cef(removed_version=13201)--*/
  virtual void SetValue(int value) = 0;
#endif

#if defined(BUILDING_CEF_SHARED) || (CEF_API_VERSION >= 13201 && CEF_API_VERSION < 13301)
  ///
  /// Set a double value.
  ///
  /*--cef(added_version=13201,removed_version=13301,capi_name=set_value2)--*/
  virtual void SetValue(double value) = 0;
#endif

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION >= 13301
  ///
  /// Set a size_t value.
  ///
  /*--cef(added_version=13301,capi_name=set_value3)--*/
  virtual void SetValue(size_t value) = 0;
#endif

  ///
  /// Returns true if a value was set.
  ///
  /*--cef()--*/
  virtual bool HasValue() = 0;
};


// In C API (auto-generated):

typedef struct _cef_test_object_t {
  ///
  /// Base structure.
  ///
  cef_base_ref_counted_t base;

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION < 13201
  ///
  /// Set an integer value.
  ///
  void(CEF_CALLBACK* set_value)(struct _cef_test_object_t* self, int value);
#else
  uintptr_t set_value_removed;
#endif

  ///
  /// Returns true (1) if a value was set.
  ///
  int(CEF_CALLBACK* has_value)(struct _cef_test_object_t* self);

  // NOTE: New function pointers are added at the end of the structure in version order.
#if defined(BUILDING_CEF_SHARED) || (CEF_API_VERSION >= 13201 && CEF_API_VERSION < 13301)
  ///
  /// Set a double value.
  ///
  void(CEF_CALLBACK* set_value2)(struct _cef_test_object_t* self, double value);
#else
  uintptr_t set_value2_removed;
#endif

  // NOTE: New function pointers are added at the end of the structure in version order.
#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION >= 13301
  ///
  /// Set a size_t value.
  ///
  void(CEF_CALLBACK* set_value3)(struct _cef_test_object_t* self, size_t value);
#endif

} cef_test_object_t;

Support removed for versions

// In C++ API:

/*--cef(source=library)--*/
class CefTestObject : public CefBaseRefCounted {
 public:
  ///
  /// Create the test object.
  ///
  /*--cef()--*/
  static CefRefPtr<CefTestObject> Create();

  // NOTE: The C API struct still needs the placeholder member for backwards compat, and
  // we still need the full version info to order properly. |capi_name| could be optional
  // for this placeholder entry.
  /*--cef(placeholder,removed_version=13201,capi_name=set_value)--*/

  // NOTE: API version check can now be simplified.
#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION < 13301
  ///
  /// Set a double value.
  ///
  /*--cef(added_version=13201,removed_version=13301,capi_name=set_value2)--*/
  virtual void SetValue(double value) = 0;
#endif

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION >= 13301
  ///
  /// Set a size_t value.
  ///
  /*--cef(added_version=13301,capi_name=set_value3)--*/
  virtual void SetValue(size_t value) = 0;
#endif

  ///
  /// Returns true if a value was set.
  ///
  /*--cef()--*/
  virtual bool HasValue() = 0;
};


// In C API (auto-generated):

typedef struct _cef_test_object_t {
  ///
  /// Base structure.
  ///
  cef_base_ref_counted_t base;

  // NOTE: Still using an opaque pointer type to reserve space at the appropriate
  // place in the structure. We could add a comment about when (which version)
  // support was removed.
  uintptr_t set_value_removed;

  ///
  /// Returns true (1) if a value was set.
  ///
  int(CEF_CALLBACK* has_value)(struct _cef_test_object_t* self);

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION < 13301
  ///
  /// Set a double value.
  ///
  void(CEF_CALLBACK* set_value2)(struct _cef_test_object_t* self, double value);
#else
  uintptr_t set_value2_removed;
#endif

#if defined(BUILDING_CEF_SHARED) || CEF_API_VERSION >= 13301
  ///
  /// Set a size_t value.
  ///
  void(CEF_CALLBACK* set_value3)(struct _cef_test_object_t* self, size_t value);
#endif

} cef_test_object_t;

[dmitry-azaraev]

I feel bit weird about examples above with ifdef magic: it should use at least ifelse or use opinionated define to hide deprecated methods (which you keep forever). But at same time - if this object sourced from CEF - i did not see any reason to be virtual at client side. Generally same rule for library side - there is no place for virtual method(s), because all such things dispatched at C ABI level.

[magreenblatt]

I feel bit weird about examples above with ifdef magic: it should use at least ifelse or use opinionated define to hide deprecated methods

Thanks for the feedback. The above examples are intentionally verbose to make understanding easier. The #define usage in the C++ API can be whatever works best/clearest and is compiler supported. The #define usage in the C API can be whatever the translator tool can deduct/generate based on required member order. We will likely pass C API header files to clang -E -DCEF_API_VERSION=<version> (e.g. the actual compiler preprocessor) before calculating the associated API hashes.

[dmitry-azaraev]

As for platform specific calls: i'm like idea when if platform did not support method - it simply did not exist in headers/metadata (so it very hard to call invalid method). But from ABI perspective i like idea to keep their method slots reserved. This allows to keep ABI definitions universal, and CEF have not so many such stuff (however it definitely exist).

Also if old method completely removed (not just deprecated and emulated) or call for method for unsupported platform is made - you should act, probably fail fast / crash. Some API in past has been completely removed, not really that happens really often, but can.

Pinning to at least major version i guess is a must-have feature, because underlying chromium behavior changes over time, and things might easily become broken. It depends on cef features in-use or web-features in-use, but i expect at least about week lag before well maintained clients got real updates to major version change. Until this it is safer to use existing milestone engine, instead of break other apps in miriad ways.

[magreenblatt]

Pinning to at least major version i guess is a must-have feature, because underlying chromium behavior changes over time, and things might easily become broken.

Agreed, for complex applications. Applications that are web-first (e.g. just a website in a container) might be fine with looser pinning since they need to keep working with older/newer Google Chrome versions in any case. In the future we might also define "API tiers" where we try to be more explicit about behavioral compatibility ranges and not just API compatibility ranges.

[magreenblatt]

• The library-side CEF C++ will have access to all API versions of client callbacks (current and deprecated). It will use the reported/selected client API version to determine which callback version to execute.
• The library-side CEF C++ will implement all API versions of methods/functions (current and deprecated). Deprecated or new versions may error out based on the reported/selected client API version.

CEF currently supports inheritance in the C API using nested structures like:

typedef struct _cef_base_object_t {
  // Base class.
  cef_base_ref_counted_t base;

  // CefBaseObject-specific function pointers here...
} cef_base_object_t ;

typedef struct _cef_inherited_object_t {
  // Base class.
  cef_base_object_t base;

  // CefInheritedObject-specific function pointers here...
} cef_inherited_object_t;

Each structure type has its own list of function pointers (see cef_base_ref_counted_t declaration here). This works fine when cef_base_object_t (and cef_base_ref_counted_t) are always compiled at a single size/version (like client-side, based on the CEF_API_VERSION value), but becomes problematic when multiple different memory layouts must be supported simultaneously (like library-side in libcef, based on the runtime-configured API version).

Proposal

To facilitate multiple version usage in the library-side of the translation layer we might want to generate flat representations of each "object" struct at each supported version. Continuing the above example we would auto-generate structs like following:

typedef struct _cef_inherited_object_13201_t {
  // CefBaseRefCounted-specific function pointers at version 13201 here...
  size_t size;
  ...

  // CefBaseObject-specific function pointers at version 13201 here...
  ...

   // CefInheritedObject-specific function pointers at version 13201 here...
  ...
} cef_inherited_object_13201_t;

// Other versions here....

And auto-generate helpers like the following to utilize the version-specific structs:

// Whatever version range uses this struct.
if (version >= 13201 && version <= 13301) {
  auto* object = new cef_inherited_object_13201_t();
  object->size = sizeof(cef_inherited_object_13201_t);
  // Assign the various member pointers here...
  object->add_ref = ...;
  object->release = ...;
  ...
  // Do something with |object|...
}
// Other version ranges here ...

This would likely be the easiest way to guarantee that libcef and the client are using the same memory layout (and the same base.size value) for a given struct at a given API version.

The library-side works with C structs in 2 ways currently:

1. The client-side provides a C struct to the library-side. Library-side then wraps that struct in a C++ object for the purpose of calling C struct function pointers from C++ methods. For example, the client returns a cef_load_handler_t* via cef_client_t::get_load_handler. This is wrapped in a CefLoadHandlerCToCpp C++ object implementing the CefLoadHandler interface. CefLoadHandlerCToCpp::OnLoadEnd inside libcef then calls cef_load_handler_t::on_load_end implemented by the client.
2. The library-side returns a C struct to the client-side. The C struct wraps an existing C++ object. For example, libcef returns a cef_browser_t* via cef_create_browser. The client calls cef_browser_t::go_back which, inside libcef (using a CefBrowserCppToC helper object), calls the underlying CefBrowser::GoBack.

With should be able to implement C++ wrapper objects for multiple versions of a C struct using a single C++ template implementation and C++20 concepts to check for and call individual struct members if they exist (like this).

Alternatives

An alternative implementation could be adding additional structure/members to the C structure to support the calculation of pointer offsets at different versions. For example, the new structure might be:

struct _child {
  // "inherited" base struct.
  struct _base {
    size_t members_size;  // sizeof(_base_members)
    struct _base_members {
       // base function ptrs here
    } members;
  } base;

  size_t members_size;  // sizeof(_child_members)
  struct _child_members {
    // child function ptrs here
  } members;
} child;

And the usage (at any version) might be:

// Offset to root (first) member type is always the same.
auto* base_members = child_ptr->base.members;
base_members->add_ref(...);

// Offset to additional member types needs to be computed based on |members_size| values.
auto* child_members = static_cast<_child_members*>(
    static_cast<uintptr_t>(child_ptr)
    + sizeof(size_t)     // sizeof(_base.members_size)
    + child_ptr->base.members_size   // version-specific sizeof(_base_members)
    + sizeof(size_t));   // sizeof(_child.members_size)
child_members->do_work(...);

This approach may generate less code and smaller binaries compared to the "Proposal" approach (due to less reliance on template specializations). However, it would be a breaking API change, more work for direct consumers of the C API to set up correctly, and (potentially) more error-prone.

[magreenblatt]

New methods are always added at the end of the class. This is relevant for the C API representation of the class as a struct of function pointers -- ordering may be applied in the translation layer.

C struct member ordering rules will be as follows:

1. All unversioned & non-experimental methods in declared order.
2. All versioned methods (with added_version metadata) grouped by version and then ordered by version (increasing) in declared order. For example, all methods with version 13101 in declared order followed by all methods with version 13201 in declared order.
3. All experimental methods (with experimental metadata) in declared order.

With this pseudo-code C++ example:

/*--cef(source=library)--*/
class CefTestObject : public CefBaseRefCounted {
 public:
  /*--cef()--*/
  virtual void UnversionedA() = 0;

#if CEF_API_VERSION >= 13201
  /*--cef(added_version=13201)--*/
  virtual void WithVersionA() = 0;
#endif

#if CEF_API_EXPERIMENTAL
  /*--cef(experimental)--*/
  virtual void ExperimentalA() = 0;
#endif

  /*--cef()--*/
  virtual void UnversionedB() = 0;

#if CEF_API_VERSION >= 13201
  /*--cef(added_version=13201)--*/
  virtual void WithVersionB() = 0;
#endif

  /*--cef()--*/
  virtual bool HasValue() = 0;

#if CEF_API_VERSION >= 13101
  /*--cef(added_version=13101)--*/
  virtual void WithVersionC() = 0;
#endif

  /*--cef()--*/
  virtual void UnversionedC() = 0;

#if CEF_API_EXPERIMENTAL
  /*--cef(experimental)--*/
  virtual void ExperimentalB() = 0;
#endif
};

The resulting C API member ordering would be:

• UnversionedA
• UnversionedB
• UnversionedC
• WithVersionC // version 13101
• WithVersionA // version 13201
• WithVersionB // version 13201
• ExperimentalA
• ExperimentalB

[magreenblatt]

Note that [next available API version] would be based on the current master/beta branch version since we don't cherry-pick new/breaking API changes to stable branches.

In the rare cases where we do add new API to stable branches, we can mark that API as experimental to avoid breaking existing fixed API versions.

[magreenblatt]

We will likely pass C API header files to clang -E -DCEF_API_VERSION=<version> (e.g. the actual compiler preprocessor) before calculating the associated API hashes.

For example:

$ cd /path/to/chromium/src/cef
$ clang -E -I. -I.. -I../out/Debug_GN_arm64/includes/cef -DUSING_CEF_SHARED -DCAPI_VERSION=12300 -DUNIT_TEST include/capi/test/cef_translator_test_capi.h > out.h

The out.h results will include data from all includes, comments are removed, and macros are expanded, so we'll need some added placeholders to use as reference points for extracting header-specific contents.

[wravery]

Just wanted to mention that I'll be following your progress here for possible integration with tauri-apps/wry. 👋🏼

There are already some experimental Rust bindings to the C API in wusyong/cef-rs. The Tauri team is interested in using CEF as another rendering option for apps that need features in Chromium that are missing in the system web views currently in use, particularly on Linux and Mac. Figuring out how to redistribute/share the CEF runtime reliably is probably the biggest open question for how to take that forward.

Thanks for tackling this!

[magreenblatt]

A first draft PR of API versioning changes now up at https://bitbucket.org/chromiumembedded/cef/pull-requests/852.

[magreenblatt]

Example of debugging API hash errors.

(for background see ApiVersioning)

1. Start with an API change that isn't correctly versioned:

% git diff --no-prefix
diff --git include/cef_audio_handler.h include/cef_audio_handler.h
index 5211cba74..91adb6062 100644
--- include/cef_audio_handler.h
+++ include/cef_audio_handler.h
@@ -106,6 +106,12 @@ class CefAudioHandler : public virtual CefBaseRefCounted {
   /*--cef()--*/
   virtual void OnAudioStreamError(CefRefPtr<CefBrowser> browser,
                                   const CefString& message) = 0;
+
+  ///
+  /// Something
+  ///
+  /*--cef()--*/
+  virtual void OnSomething() = 0;
 };

2. Run version_manager check (or cef_create_projects), see that it fails:

% python3 tools/version_manager.py -c --fast-check                                            
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi_versions.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/cpptoc/audio_handler_cpptoc.cc
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.cc
Done translating - Wrote 5 files.
Hashes for experimental version are unchanged.
Hashes for next version are unchanged.
No hash updates required.
ERROR: Hashes for version 13304 do not match!
ERROR: Hashes for version 13300 do not match!
ERROR: 2 hashes checked and failed
0 hashes checked and match (0/5 versioned, 0/2 untracked).

3. To analyze the failure run version_manager check both before and after the change with debugging enabled:

% git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
	modified:   include/cef_audio_handler.h

% git stash
Saved working directory and index state

% python3 tools/version_manager.py -c --fast-check --debug-dir=/Users/marshall/tmp/debug_before
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi_versions.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/cpptoc/audio_handler_cpptoc.cc
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.cc
Done translating - Wrote 5 files.
Updating hashes for experimental version.
Updating hashes for next version.
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/cef_api_untracked.json
2 hashes checked and match (2/5 versioned, 0/2 untracked).

% git stash pop                                                                                
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
	modified:   include/cef_audio_handler.h

% python3 tools/version_manager.py -c --fast-check --debug-dir=/Users/marshall/tmp/debug_after 
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi_versions.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/cpptoc/audio_handler_cpptoc.cc
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.cc
Done translating - Wrote 5 files.
Updating hashes for experimental version.
Updating hashes for next version.
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/cef_api_untracked.json
ERROR: Hashes for version 13304 do not match!
ERROR: Hashes for version 13300 do not match!
ERROR: 2 hashes checked and failed
0 hashes checked and match (0/5 versioned, 0/2 untracked).

4. Compare API hash intermediate output (objects.txt) for the impacted API version (e.g. 13300):

% git diff --word-diff /Users/marshall/tmp/debug_before/13300/objects.txt /Users/marshall/tmp/debug_after/13300/objects.txt

5. Fix the API versioning:

% git diff --no-prefix           
diff --git include/cef_audio_handler.h include/cef_audio_handler.h
index 5211cba74..f29538c12 100644
--- include/cef_audio_handler.h
+++ include/cef_audio_handler.h
@@ -106,6 +106,14 @@ class CefAudioHandler : public virtual CefBaseRefCounted {
   /*--cef()--*/
   virtual void OnAudioStreamError(CefRefPtr<CefBrowser> browser,
                                   const CefString& message) = 0;
+
+#if CEF_API_ADDED(CEF_NEXT)
+  ///
+  /// Something
+  ///
+  /*--cef(added=next)--*/
+  virtual void OnSomething() = 0;
+#endif
 };

6. Verify that the fix is correct:

% python3 tools/version_manager.py -c --fast-check                                            
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi_versions.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/cpptoc/audio_handler_cpptoc.cc
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.cc
Done translating - Wrote 5 files.
Hashes for experimental version are unchanged.
Hashes for next version are unchanged.
No hash updates required.
2 hashes checked and match (2/5 versioned, 0/2 untracked).

[magreenblatt]

Example of adding a new API version.

(for background see ApiVersioning)

1. Make an API change using the CEF_NEXT placeholder:

% git diff --no-prefix           
diff --git include/cef_audio_handler.h include/cef_audio_handler.h
index 5211cba74..f29538c12 100644
--- include/cef_audio_handler.h
+++ include/cef_audio_handler.h
@@ -106,6 +106,14 @@ class CefAudioHandler : public virtual CefBaseRefCounted {
   /*--cef()--*/
   virtual void OnAudioStreamError(CefRefPtr<CefBrowser> browser,
                                   const CefString& message) = 0;
+
+#if CEF_API_ADDED(CEF_NEXT)
+  ///
+  /// Something
+  ///
+  /*--cef(added=next)--*/
+  virtual void OnSomething() = 0;
+#endif
 };

2. Run version_manager add, get notified of a problem:

% python3 tools/version_manager.py -a             
ERROR: NEXT usage found in CEF headers:

include/cef_audio_handler.h:110:#if CEF_API_ADDED(CEF_NEXT)
include/cef_audio_handler.h:114:  /*--cef(added=next)--*/

Fix manually or run with --replace-next.

3. Run again with --replace-next to substitute CEF_NEXT usage with the new version number and add a new cef_api_versions.json entry:

% python3 tools/version_manager.py -a --replace-next
Attempting to replace NEXT usage with 13305 in CEF headers:

include/cef_audio_handler.h:110:#if CEF_API_ADDED(CEF_NEXT)
include/cef_audio_handler.h:114:  /*--cef(added=next)--*/

For file /Users/marshall/code/chromium_git/chromium/src/cef/include/cef_audio_handler.h:
  Replaced 2 of 2 NEXT instances

All NEXT instances successfully replaced.
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/include/capi/cef_audio_handler_capi_versions.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/cpptoc/audio_handler_cpptoc.cc
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.h
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/libcef_dll/ctocpp/audio_handler_ctocpp.cc
Done translating - Wrote 5 files.
Hashes for experimental version are unchanged.
Hashes for next version are unchanged.
Adding hashes for version 13305.
Writing file /Users/marshall/code/chromium_git/chromium/src/cef/cef_api_versions.json
6 hashes checked and match (6/6 versioned, 0/2 untracked).

4. Manually inspect the changes:

% git diff --no-prefix
diff --git cef_api_versions.json cef_api_versions.json
index aded39881..787c891d1 100644
--- cef_api_versions.json
+++ cef_api_versions.json
@@ -34,8 +34,15 @@
       "mac": "405810f1f8b146678867b6a91cbe8c4670febbbf",
       "universal": "be55f7cf1813ae098d4f68b2a2c9ca85784fc3ee",
       "windows": "73bb28a92f4be742e3fc80057a80797e3bf23063"
+    },
+    "13305": {
+      "comment": "Added January 13, 2025.",
+      "linux": "e9f973a34c5a548ae7e4f21daeb77705549b91e3",
+      "mac": "28cfdbaae0fb370ecc0ad546ece4a735f0d2883a",
+      "universal": "50bcb922edd9b7360deb74ec27cc318ebcdfa63d",
+      "windows": "09a55841b4299c4a349ade7fb1a3cdaca607fc60"
     }
   },
-  "last": "13304",
+  "last": "13305",
   "min": "13300"
 }
\ No newline at end of file
diff --git include/cef_audio_handler.h include/cef_audio_handler.h
index 5211cba74..e1773bee9 100644
--- include/cef_audio_handler.h
+++ include/cef_audio_handler.h
@@ -106,6 +106,14 @@ class CefAudioHandler : public virtual CefBaseRefCounted {
   /*--cef()--*/
   virtual void OnAudioStreamError(CefRefPtr<CefBrowser> browser,
                                   const CefString& message) = 0;
+
+#if CEF_API_ADDED(13305)
+  ///
+  /// Something
+  ///
+  /*--cef(added=13305)--*/
+  virtual void OnSomething() = 0;
+#endif
 };

5. Verify that no further updates are required:

% python3 tools/version_manager.py -a               
Hashes for experimental version are unchanged.
Hashes for next version are unchanged.
Hashes for last version 13305 are unchanged.
No hash updates required.
6 hashes checked and match (6/6 versioned, 0/2 untracked).

[magreenblatt]

Initial documentation for API versioning has been added at https://bitbucket.org/chromiumembedded/cef/wiki/ApiVersioning

[magreenblatt]

Remaining work related to API versioning:

• Add support for API versioning of platform-specific header files. This involves running clang with a configuration independent of the host system (e.g. to process windows-specific headers on mac/linux, and similar). Will hash a single "default" configuration for each platform (for example, x64 + X11 on Linux).
• Remove UNIVERSAL hash and rely solely on platform-specific hashes which track the complete API contents for that platform.

[ShelleyLake]

Naive question. Will OSR be broken if using WebView2? It seems that Microsoft is quite lazy to implement it. https://github.com/MicrosoftEdge/WebView2Feedback/issues/547

[magreenblatt]

@ShelleyLake Do CEF and WebView2 work in the same app currently? If so, this issue should not impact it.

[ShelleyLake]

@magreenblatt - I fear I misunderstood your intention. Sorry. Pls forget my comment :-)

[chriscurtismozenda]

Hey @magreenblatt, I am trying to run my existing app with the following build: cef_binary_133.4.3+gff061be+chromium-133.0.6943.35_windows32_beta_minimal. To this point it has been running fine with many previous versions and most recently it is using cef_binary_132.3.1+g144febe+chromium-132.0.6834.83_windows32_beta_minimal

The problem I am facing is that regardless of the version (999998, 999999, or 13304) that I place in the CEF_API_VERSION project configuration variable, CEF is crashing when trying to execute line 97 of \cef\libcef\browser\prefs\pref_registrar.cc

void RegisterCustomPrefs(cef_preferences_type_t type, PrefRegistrySimple* registry) { if (auto app = CefAppManager::Get()->GetApplication()) { if (auto handler = app->GetBrowserProcessHandler()) { CefPreferenceRegistrarImpl registrar(registry); handler->OnRegisterCustomPreferences(type, &registrar); } } }

It seems that the crash is happening because I haven't implemented the 'OnRegisterCustomPreferences' event for processHandler in my application. Is it a requirement that I do this now moving forward?

If I go ahead and implement that one then it moves past this point but begins crashing on other "handlers" and/or events of those handlers that I haven't implemented such as CfxResourceBundleHandler, CfxKeyboardHandler, CfxFocusHandler, etc. Is this expected as a result of these changes?

Before I go through and implement every single event of every handler just to tell it to use the default behavior I wanted to be sure. I'm guessing I'm doing something wrong or don't understand something, so I thought I'd ask first.

Thank you for your time.

[magreenblatt]

@chriscurtismozenda

1. What compiler are you using?
2. Are you using the CMake configuration provided with the binary distribution?
3. If you build a sample app (cefclient/cefsimple) using the binary distribution does is behave as expected?
4. Did you rebuild/re-link libcef_dll_wrapper after the update for your application?
5. Does it help if you don't explicitly configure CEF_API_VERSION?

[chriscurtismozenda]

I'm a .Net c# developer mostly, but I'll try and answer your questions the best I can. I apologize in advance for my lack of knowledge in this area. I am using a parsing tool to create a wrapper for CEF in C#. Similar to what was done by ChromiumFX previously.

1. What compiler are you using? MSVC. I am compiling via VS 2022 with a toolset version of 143
2. Are you using the CMake configuration provided with the binary distribution? No. I am not compiling CEF myself at all. Just using the distributions available here: https://cef-builds.spotifycdn.com/index.html#windows32
3. If you build a sample app (cefclient/cefsimple) using the binary distribution does is behave as expected? Haven't done that and don't know how to. If this doesn't require a lot of c++ knowledge I'd be happy to try if you can point me towards some documentation.
4. Did you rebuild/re-link libcef_dll_wrapper after the update for your application? As much as I have ever had to do when updating to previous versions. I've been following the same pattern now for quite a few years.
5. Does it help if you don't explicitly configure CEF_API_VERSION? No, it acts the same from what I can tell regardless of the three versions I specified above. But, if I put a nonsensical version in there then it fails to compile or run so I know its doing something. If I don't put a CEF_API_VERSION in there at all then it behaves the same way and fails first when trying to register custom preferences.

Hope some of that helps.

[magreenblatt]

@chriscurtismozenda

I am using a parsing tool to create a wrapper for CEF in C#

I suggest you contact the author of that tool (if you didn't write it yourself) and see if it needs to be updated.

[chriscurtismozenda]

@chriscurtismozenda

I am using a parsing tool to create a wrapper for CEF in C#

I suggest you contact the author of that tool (if you didn't write it yourself) and see if it needs to be updated.

Again, my apologies for not being better informed. You are stating that I should tell the developer of the parser/wrapper that it should require some changes to continue to function as it did before without any downstream changes in my c# application? E.g., we shouldn't have to implement any additional handlers or events for it to continue to work as it did? Is all the information the developer needs in the documentation listed here?

https://bitbucket.org/chromiumembedded/cef/wiki/ApiVersioning

[magreenblatt]

@chriscurtismozenda

You are stating that I should tell the developer of the parser/wrapper that it should require some changes to continue to function as it did before

Potentially yes.

without any downstream changes in my c# application? E.g., we shouldn't have to implement any additional handlers or events for it to continue to work as it did?

That would be up to the C# parser/wrapper developer, but I would presume so.

Is all the information the developer needs in the documentation listed here?

Yes, or in this issue.

[magreenblatt]

We're currently ironing out some API gen issues that necessitate API hash changes for existing API versions on the M133/134 release branches. One symptom of this will be the bugfix version component occasionally resetting on the automated builders (e.g. going from 133.4.6 back to 133.4.2). This is a temporary issue, and once the API gen changes are complete the API version hashes (and bugfix version numbers) should be stable going forward.

Related PR: https://bitbucket.org/chromiumembedded/cef/pull-requests/869

[magreenblatt]

With API versioning support now looking pretty solid we're going to move on to the next Phase.

Shared runtime behavior

This will involve at least the following:

• Use a non-generic (app-specific, like exe name) default value for CefSettings.root_cache_path.
• Support use of a versioned (or at least unbundled) CEF Framework on macOS. See this thread for signing/notarization considerations. There may also be path-related considerations in CEF/Chromium.
• Support use of dynamic loading on Windows. For example, if we find the shared CEF binaries by reading registry entries.

[hoshinolina]

Are there any recommendations for packaging CEF as a systemwide installation on Linux? I'm working on a Fedora package, and currently it looks like this:

• Binaries/resources/etc go in /usr/lib64/cef. This is a shared install among all possible consumers.
• For building, I have cef-apiXXXXX-devel packages which can be installed. They provide the precompiled libcef_dll_wrapper.a for that API version in /usr/lib64/cef, as well as as the includes in /usr/include/cef/ (e.g. /usr/include/cef/cef_base.h). To build a consumer, you have to install the appropriate (mutually exclusive) devel package.
• I inject a check at the end of cef/cef_api_hash.h to test that CEF_API_VERSION is defined properly (matching the devel package installed, which has the precompiled wrapper for that api version).

The ugliest thing right now, besides the multiple builds of the wrapper/include stuff, is that CEF includes are supposed to use the format <include/cef_foo.h>, which is kind of weird (normally the include component is part of the system include path). For a systemwide install I turn those into cef/cef_foo.h but that means I have to patch all the include files, as well as any consumers that expect include/ paths. Are there any plans to move CEF to a more standard include hierarchy so the headers can be installed systemwide?

Any comments on this approach would be welcome!