Horreum icon indicating copy to clipboard operation
Horreum copied to clipboard

Datastores can return large numbers runs that are processing synchronously

Open johnaohara opened this issue 8 months ago • 0 comments

Describe the bug

Some backend datastore integrations can return a large number of runs.

This can cause issues with importing a lot of run data in one query, such as long transaction times, failure to process a single run data will fail the entire import etc.

The processing of run data should be performed asynchronously in these cases and runs should be offloaded to an external queue.

See https://github.com/Hyperfoil/Horreum/pull/1703 for an example datastore that can return a large number of run results

johnaohara avatar Jul 03 '24 12:07 johnaohara