cc-index-table icon indicating copy to clipboard operation
cc-index-table copied to clipboard

Published 20 hours ago • verified publisher icon commoncrawl

News and wat wet compatibilities

Open jt55401 opened this issue 8 months ago • 2 comments

Modifications to the index-table routines to:

  1. Run more easily locally (devcontainer and plain dockerfile)
  2. Make the logic which parses the filenames in the cdxj so it's compatible with wat, wet, and news paths
  3. Add a test case for the above

jt55401 avatar May 31 '24 01:05 jt55401