archive-query-log icon indicating copy to clipboard operation
archive-query-log copied to clipboard

Published 20 hours ago • verified publisher icon webis-de

Defragment S3 WARC storage

Open janheinrichmerker opened this issue 1 year ago • 1 comments
trafficstars

Look for "smaller" WARC files on S3 that can be merged.

janheinrichmerker avatar Feb 13 '24 12:02 janheinrichmerker

@Simrut I feel this is no longer urgent, as the S3 upload seems to run fine and no longer creates many small files. Should we close this issue for now?

janheinrichmerker avatar Apr 24 '25 09:04 janheinrichmerker