TheHive4py icon indicating copy to clipboard operation
TheHive4py copied to clipboard

Case creation with bulk insert of observables

Open robomotic opened this issue 5 years ago • 6 comments

(Request Type)

Request Type

Feature Request

Problem Description

When creating a new case, all the observables needs to be added one to one. This is fine for a few hundred but it becomes super slow when inserting 10k or more. I typically submit 50k-60k of observables for each case and as you can imagine it takes a long time.

Steps to Reproduce

  1. Create a new case
  2. Add 50k observables or more
  3. Measure time to completion

robomotic avatar Apr 07 '20 22:04 robomotic

Hello @robomotic

  • what do you mean by it takes a long time?
  • are your observables of the same type?

nadouani avatar May 24 '20 21:05 nadouani

For me it seems to take about 1 second per observable when creating a hive case. Is this normal? A couple hundred observables takes on the order of minutes.

scottbelden avatar Aug 03 '21 19:08 scottbelden

Yes that was approx the same time it was taking, I am trying to use Hive4 but stuck with a docker problem as per my other issue.

robomotic avatar Aug 03 '21 23:08 robomotic

I tested with creating a thread pool to do the bulk insert of observables and it worked very nicely. I didn't measure the speed (though perhaps I will and report back), but it seemed to be on the order of 10-20 observables a second rather than 1 per second. I'll try to put together a PR since it should be pretty easy to do so.

scottbelden avatar Aug 04 '21 17:08 scottbelden

Question is this with the latest version of TheHive4 ? Can you post your configuration? Are you using one of the docker templates or bare install ?

robomotic avatar Aug 04 '21 17:08 robomotic

I am using a bare install of 1.8.1 (the most recent version)

scottbelden avatar Aug 04 '21 17:08 scottbelden