src7

Thank you, we tried on different servers and `Processed pastes` is stuck at about 150 (on one single feeder, using `bin/import_dir.py`)

Thank you, currently I am trying something with `configs/6382.conf`

Hi, Queues are not empty, but not stuck. Files processed are regular pastes from pastebin. The disk is NVME or RAM based (direct I/O disabled for the last one). I...

Imported pastes are not compressed (if it can help)

Yes, and that is not the problem. I see way more than 150 pastes per minute imported in the console (no sleep set).

1. `import_dir` (also tested with different sleep values and even without the sleep line) 2. In the Web Dashboard, `Processed pastes`

After some research I have found that tuning this delay changes everything : https://github.com/CIRCL/AIL-framework/blob/b4a85c0e9808c1660adc36400e33238121099f4d/bin/Helper.py#L105 Currently reaching 4000 pastes per minute and working on more.

Hello, For now I use this workaround to rename `to_import/1970/01/01/api_scrape_item.php?i=aZe0Rt1Y` to `to_import/1970/01/01/aZe0Rt1Y.txt` in batch in a very efficient way. Command (in the `to_import` folder) `mmv ';*\=*' '#1#3.txt'` (use -n to...

Hi, what about the `?` and the `=` ?

Nice (it is a bit extreme but two files named &.txt and ?.txt can't coexist then ? Not a problem for me)