Greynoise empty error

[drosetti]

What happened

In the advanced search page I noticed some failed query with a 400. The problem is greynoise error with an empty string.

Environment

1. OS: Ubuntu 24.04
2. IntelOwl version: 6.3.1

What did you expect to happen

Get a real error

How to reproduce your issue

Difficult, It seems a rate limit problem, I suggest to do multiple request at the same time and hope some of them fails.

Error messages and logs

[ggrayapura7]

The issue seems to be caused by an empty query being sent to GreyNoise, which results in a 400 error. It could also be related to rate limiting. A possible fix would be to add input validation to prevent empty queries from being sent and implement proper error handling for failed requests. Additionally, checking and handling GreyNoise's rate limits with retries or exponential backoff might help resolve this issue.

[drosetti]

Yes this could be the root cause and I agree that it's import to work on it.

A possible fix would be to add input validation to prevent empty queries from being sent and implement proper error handling for failed requests

When API to start a job is used, there is a validation about the analyzable's value to use. Adding a validation here is redundant.

checking and handling GreyNoise's rate limits with retries or exponential backoff might help resolve this issue.

This is more interesting imho: in case the rate limit is very low, we could try multiple attempts inside this analyzer. In case the rate limit is calculated based on hour or days, this don't solve the problem.

However I think there is another bug: an empty string is added to the error list, and this shouldn't happen.

[ggrayapura7]

Thank you for the insights!

I’ll review the existing validation logic to ensure that adding another check isn’t redundant. Regarding GreyNoise’s rate limits, I’ll explore implementing retries with exponential backoff. I understand that if the rate limits are set on an hourly/daily basis, simple retries won’t be effective, so I’ll look into possible workarounds. I’ll also investigate the issue with the empty string being added to the error list and try to identify the root cause. Since I’m still learning, I might have some questions along the way. I appreciate your guidance and will update you as I make progress. Let me know if there are any specific areas I should focus on!

[github-actions[bot]]

This issue has been marked as stale because it has had no activity for 10 days. If you are still working on this, please provide some updates.