connectors icon indicating copy to clipboard operation
connectors copied to clipboard

[Shodan] (Most) indicators that are enriched create Python stack traces

Open Kalkran opened this issue 1 year ago • 3 comments

Description

As raised on Slack: https://filigran-community.slack.com/archives/C06CF1N302W/p1725539276021819

The Shodan enricher is set with CONNECTOR_SCOPE=IPv4-Addr,Indicator and CONNECTOR_AUTO=true. This means all new Indicators will be handled by the Shodan connector. Unfortunately, all stix patterned Indicators, which is most indicators, raise a ValueError and accompanying partial stack trace.

It is probably desirable to quiet these exceptions.

https://github.com/OpenCTI-Platform/connectors/blob/master/internal-enrichment/shodan/src/shodanImport.py#L524-L527

Environment

Currently on 6.2.18

Reproducible Steps

Run the Shodan connector with the default compose file and add a new indicator (or enrich an old indicator) which is not of pattern_type: shodan.

Expected Output

No errors

Actual Output

Errors image

Additional information

This was probably added in #963 and previously mentioned in #2523

Screenshots (optional)

Kalkran avatar Sep 05 '24 14:09 Kalkran

@Megafredo: what can we do to not generate these traces? In addition, we must work on a readme for this connector. Let's grab some time to do this together.

Jipegien avatar Sep 09 '24 07:09 Jipegien

Hi @Kalkran, Your observation is correct, if you set the ‘auto’ environment variable to true, you risk getting a set of traceback errors of the following type: (ValueError : Unsupported pattern type : ‘patternType’) due to the indicator type events being triggered. However, there is a small function directly accessible on OpenCTI which allows you to filter the indicator pattern type :

In the connector Shodan page: Data / Ingestion / Connectors / Shodan

Use the added filters entitled ‘Trigger filters’, as shown in the image. Add two filters: Pattern type = shodan OR Entity type = IPv4 address

And now you shouldn't have any problems with tracebacks for other pattern type indicators ! trigger_filters

Megafredo avatar Sep 11 '24 06:09 Megafredo

Was just looking at this, I changed the code to allow stix at https://github.com/OpenCTI-Platform/connectors/blob/dd3f115c5427ec1c924616d7222519a35c1265fd/internal-enrichment/shodan/src/shodanImport.py#L422 But there is another issue, when it does the faucet search, it looks that the api does not support passing it a pattern, I never get any results otherwise. https://github.com/OpenCTI-Platform/connectors/blob/dd3f115c5427ec1c924616d7222519a35c1265fd/internal-enrichment/shodan/src/shodanImport.py#L447

annoyingapt avatar Sep 11 '24 07:09 annoyingapt

I close the issue for now but it can be re opened if needed

helene-nguyen avatar Jan 06 '25 11:01 helene-nguyen