bff icon indicating copy to clipboard operation
bff copied to clipboard

Published 20 hours ago • verified publisher icon allenai

Barebones python/S3 implementation

Open revbucket opened this issue 4 months ago • 0 comments

Added bff_v0.py which is a simple python script to:

  1. download all .jsonl.gz's from a specified S3 directory
  2. Run BFF on ^
  3. Upload the outputs back to S3

Also modified the main.rs to use the faster init for bloom filters

revbucket avatar Feb 29 '24 21:02 revbucket