warcbase
warcbase copied to clipboard
Build CLI interface for admin metadata table
Basic idea is to have a warcbase.meta table for storing collection-level metadata, e.g.,
- the Lucene FST for mapping URL <-> id
- record of data ingestion
- ARC/WARC
- etc.
This is a start: https://github.com/lintool/warcbase/blob/master/src/main/java/org/warcbase/WarcbaseAdmin.java