cJSON under TrustInSoft CI

[jakub-zwolakowski]

Hi,

I initiated the configuration of cJSON on the new tool TrustInSoft CI. It's a source code analyzer, which analyzes execution paths (usually unit tests in your repo) to detect Undefined Behaviors along the way. Coverage includes all the defects that sanitizers ASAN, UBSan and MSan find, plus a large number of other (usually subtle) defects.

I've set up TrustInSoft CI both on your test suite located in the tests directory and on the fuzzing inputs in the fuzzing/inputs directory. You can check the results here. The tool has proved the absence of Undefined Behaviors on all the analyzed execution paths. Since it relies on formal methods, it is able to detect the most subtle Undefined Behaviors, so these results are particularly impressive!

As I've set up TrustInSoft CI on my fork, one next step would be that you try it out in a continuous integration mode.

May I ask if this is something that you'd be interested in?

The setup consists in writing a configuration file. I can put my initial setup in a PR if you wish me to.

Thanks!

[Alanscut]

Hi @jakub-zwolakowski , thank you for intruducing this is an amazing tool, and it certainly make sense. However, I have no permission to configure this tool. I have configured the github actions CI for it, and cJSON has also been integrated in oss fuzz, do you think these can replace trust CI?

[jakub-zwolakowski]

Hi @Alanscut,

Yes, these results are on my fork of cJSON and on my TIS CI account. That's why you don't have permission to change the configuration.

In order to configure the tool, you'd need to do two things:

• Add a configuration file to the project (I'll make a PR with my configuration, so you can see how it looks like and merge if it suits you).
• Create a TIS CI account (for free) and add the cJSON project to TIS CI (I'll describe the steps in the PR).

About your other question, fuzzing complements TIS CI so it does not replace it:

• TIS CI is a static analyzer working directly on the source code. It relies on formal methods so it's very precise: it can find ALL undefined behaviors on the analyzed execution paths (e.g. the tests in your repo).
• Tools like sanitizers + OSS fuzz are faster but less precise, so they can analyze more execution paths but they only detect some undefined behaviors and give you no mathematical guarantee whatsoever.

In short, both tools can help you find more undefined behaviors and TIS CI gives you this mathematical guarantee of the absence of undefined behaviors on your test suite.

Let me know if that clarifies!