MISP API output bot `significant_fields` does work correctly for multiple fields

[karsa-mistmere]

When multiple fields are listed in the MISPAPIOutputBot's significant_fields setting, the bot uses PyMISP’s build_complex_query to construct a query (e.g., {'AND': ['12.123.21.21', 'foobar.com']}) and passes it to the value argument of PyMISP.search().

However, value does not support complex queries.

Expected behaviour

The bot should correctly filter MISP events based on multiple significant fields.

Actual behaviour

Since value does not interpret complex queries, the filter is ignored, causing the bot to return all events. This leads to incorrect duplicate event detection.

Supporting Evidence

According to PyMISP's official documentation, build_complex_query can be used with tags and event_tags, but not value. Testing confirms that complex queries are not processed as expected.

TL;DR

The MISPAPIOutputBot's significant_fields setting only works correctly with a single field or when multiple fields never appear together in the same event.

Updates

Edit: seems like, while the feature itself is undocumented, this might be an error on MISP's side. The complex value is actually parsed in Event::set_filter_value, but results in an entirely erroneous condition, see https://github.com/MISP/MISP/issues/10250

[sebix]

The MISP API output bot was written and contributed by @bernhardreiter , can you help here?

[bernhardreiter]

The contribution of the MISP API bot was the result of a contract. I did test options way back then and the customer also did, so I assume that we also tested that particular behaviour. The situation now is in line with the tentative analysis in https://github.com/MISP/MISP/issues/10250 that the MISP behaviour could be a regression. Anyway it seems worth waiting for the MISP's side take on or improvement of the problem.

[sebix]

I didn't notice the activity in MISP itself, so I agree to wait for the changes there.

Thank you @karsa-mistmere for analyzing the problem in such detail.