Tool to detect API/ABI non-compliant changes, and integrate with CI.

[lalitb]

As discussed in today's Feb 22, 2023 community meeting:

• Need tooling to detect ABI changes in CI, instead of relying on code review only
• There was some research done earlier on one of the tools- ABI Compliance Checker (lvc.github.io), but good to compare other alternatives before finalizing.
• The scope of the validation is only the API change, and otel-cpp doesn't guarantee ABI compatibility for SDK code.

[marcalff]

I have been playing a bit with this, to see what the tooling can do.

Preliminary results below.

Data collected:

-rw-r--r-- 1 malff users  310366 Feb 24 18:10 abi_check_baggage-clang13-local-latest.dump
-rw-r--r-- 1 malff users  310034 Feb 24 18:09 abi_check_baggage-clang13-local-v1.0.0.dump
-rw-r--r-- 1 malff users  310034 Feb 24 18:09 abi_check_baggage-clang13-local-v1.1.0.dump
-rw-r--r-- 1 malff users  310034 Feb 24 18:09 abi_check_baggage-clang13-local-v1.2.0.dump
-rw-r--r-- 1 malff users  310034 Feb 24 18:10 abi_check_baggage-clang13-local-v1.3.0.dump
-rw-r--r-- 1 malff users  310365 Feb 24 18:10 abi_check_baggage-clang13-local-v1.4.0.dump
-rw-r--r-- 1 malff users  310365 Feb 24 18:10 abi_check_baggage-clang13-local-v1.5.0.dump
-rw-r--r-- 1 malff users  310365 Feb 24 18:10 abi_check_baggage-clang13-local-v1.6.0.dump
-rw-r--r-- 1 malff users  310365 Feb 24 18:10 abi_check_baggage-clang13-local-v1.7.0.dump
-rw-r--r-- 1 malff users  310365 Feb 24 18:10 abi_check_baggage-clang13-local-v1.8.0.dump
-rw-r--r-- 1 malff users 1029899 Feb 24 18:11 abi_check_baggage-gcc12-local-latest.dump
-rw-r--r-- 1 malff users 1022781 Feb 24 18:10 abi_check_baggage-gcc12-local-v1.0.0.dump
-rw-r--r-- 1 malff users 1022781 Feb 24 18:10 abi_check_baggage-gcc12-local-v1.1.0.dump
-rw-r--r-- 1 malff users 1022781 Feb 24 18:10 abi_check_baggage-gcc12-local-v1.2.0.dump
-rw-r--r-- 1 malff users 1022781 Feb 24 18:11 abi_check_baggage-gcc12-local-v1.3.0.dump
-rw-r--r-- 1 malff users 1028139 Feb 24 18:11 abi_check_baggage-gcc12-local-v1.4.0.dump
-rw-r--r-- 1 malff users 1028139 Feb 24 18:11 abi_check_baggage-gcc12-local-v1.5.0.dump
-rw-r--r-- 1 malff users 1028139 Feb 24 18:11 abi_check_baggage-gcc12-local-v1.6.0.dump
-rw-r--r-- 1 malff users 1028133 Feb 24 18:11 abi_check_baggage-gcc12-local-v1.7.0.dump
-rw-r--r-- 1 malff users 1028133 Feb 24 18:11 abi_check_baggage-gcc12-local-v1.8.0.dump
-rw-r--r-- 1 malff users  502070 Feb 24 18:10 abi_check_trace-clang13-local-latest.dump
-rw-r--r-- 1 malff users  486941 Feb 24 18:09 abi_check_trace-clang13-local-v1.0.0.dump
-rw-r--r-- 1 malff users  487496 Feb 24 18:09 abi_check_trace-clang13-local-v1.1.0.dump
-rw-r--r-- 1 malff users  487496 Feb 24 18:09 abi_check_trace-clang13-local-v1.2.0.dump
-rw-r--r-- 1 malff users  487496 Feb 24 18:10 abi_check_trace-clang13-local-v1.3.0.dump
-rw-r--r-- 1 malff users  487496 Feb 24 18:10 abi_check_trace-clang13-local-v1.4.0.dump
-rw-r--r-- 1 malff users  487496 Feb 24 18:10 abi_check_trace-clang13-local-v1.5.0.dump
-rw-r--r-- 1 malff users  487496 Feb 24 18:10 abi_check_trace-clang13-local-v1.6.0.dump
-rw-r--r-- 1 malff users  488967 Feb 24 18:10 abi_check_trace-clang13-local-v1.7.0.dump
-rw-r--r-- 1 malff users  488967 Feb 24 18:10 abi_check_trace-clang13-local-v1.8.0.dump
-rw-r--r-- 1 malff users 1200288 Feb 24 18:11 abi_check_trace-gcc12-local-latest.dump
-rw-r--r-- 1 malff users 1170334 Feb 24 18:10 abi_check_trace-gcc12-local-v1.0.0.dump
-rw-r--r-- 1 malff users 1171009 Feb 24 18:10 abi_check_trace-gcc12-local-v1.1.0.dump
-rw-r--r-- 1 malff users 1171009 Feb 24 18:10 abi_check_trace-gcc12-local-v1.2.0.dump
-rw-r--r-- 1 malff users 1171009 Feb 24 18:11 abi_check_trace-gcc12-local-v1.3.0.dump
-rw-r--r-- 1 malff users 1171009 Feb 24 18:11 abi_check_trace-gcc12-local-v1.4.0.dump
-rw-r--r-- 1 malff users 1171009 Feb 24 18:11 abi_check_trace-gcc12-local-v1.5.0.dump
-rw-r--r-- 1 malff users 1171009 Feb 24 18:11 abi_check_trace-gcc12-local-v1.6.0.dump
-rw-r--r-- 1 malff users 1173366 Feb 24 18:11 abi_check_trace-gcc12-local-v1.7.0.dump
-rw-r--r-- 1 malff users 1173366 Feb 24 18:11 abi_check_trace-gcc12-local-v1.8.0.dump

For GCC12:

[email protected]:compat_reports> grep -R -l "kind:binary;verdict:incompatible" * | grep gcc12 | sort
abi_check_trace-gcc12-local/v1.0.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.1.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.2.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.3.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.4.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.5.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.6.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.0.0_to_v1.8.0/compat_report.html

[email protected]:compat_reports> grep -R -l "kind:binary;verdict:compatible" * | grep gcc12 | sort 
abi_check_baggage-gcc12-local/v1.0.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.1.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.2.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.3.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.4.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.5.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.6.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.0.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.2.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.3.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.4.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.5.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.6.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.1.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_v1.3.0/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_v1.4.0/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_v1.5.0/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_v1.6.0/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.2.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.3.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.3.0_to_v1.4.0/compat_report.html
abi_check_baggage-gcc12-local/v1.3.0_to_v1.5.0/compat_report.html
abi_check_baggage-gcc12-local/v1.3.0_to_v1.6.0/compat_report.html
abi_check_baggage-gcc12-local/v1.3.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.3.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.4.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.4.0_to_v1.5.0/compat_report.html
abi_check_baggage-gcc12-local/v1.4.0_to_v1.6.0/compat_report.html
abi_check_baggage-gcc12-local/v1.4.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.4.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.5.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.5.0_to_v1.6.0/compat_report.html
abi_check_baggage-gcc12-local/v1.5.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.5.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.6.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.6.0_to_v1.7.0/compat_report.html
abi_check_baggage-gcc12-local/v1.6.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.7.0_to_latest/compat_report.html
abi_check_baggage-gcc12-local/v1.7.0_to_v1.8.0/compat_report.html
abi_check_baggage-gcc12-local/v1.8.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.2.0/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.3.0/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.4.0/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.5.0/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.6.0/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.1.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_v1.3.0/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_v1.4.0/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_v1.5.0/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_v1.6.0/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.2.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.3.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.3.0_to_v1.4.0/compat_report.html
abi_check_trace-gcc12-local/v1.3.0_to_v1.5.0/compat_report.html
abi_check_trace-gcc12-local/v1.3.0_to_v1.6.0/compat_report.html
abi_check_trace-gcc12-local/v1.3.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.3.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.4.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.4.0_to_v1.5.0/compat_report.html
abi_check_trace-gcc12-local/v1.4.0_to_v1.6.0/compat_report.html
abi_check_trace-gcc12-local/v1.4.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.4.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.5.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.5.0_to_v1.6.0/compat_report.html
abi_check_trace-gcc12-local/v1.5.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.5.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.6.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.6.0_to_v1.7.0/compat_report.html
abi_check_trace-gcc12-local/v1.6.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.7.0_to_latest/compat_report.html
abi_check_trace-gcc12-local/v1.7.0_to_v1.8.0/compat_report.html
abi_check_trace-gcc12-local/v1.8.0_to_latest/compat_report.html

[ThomsonTan]

Does this mean it is always recommended for the user to build with the same version of API and SDK?

otel-cpp doesn't guarantee ABI compatibility for SDK code

[lalitb]

Does this mean it is always recommended for the user to build with the same version of API and SDK?

No. This actually means the SDK specific component (e.g. Processor and Exporter Interfaces) can be modified in ABI breaking way. So people having external exporters or processors are expected to rebuild them to be compatible with new SDK version.

[github-actions[bot]]

This issue was marked as stale due to lack of activity.

[marcalff]

Tooling added in a dedicated repository:

• https://github.com/marcalff/opentelemetry-cpp-abi

[owent]

Could we publish these reports to somewhere?

[marcalff]

Could we publish these reports to somewhere?

Not yet, but will look into this.