wikijump
wikijump copied to clipboard
[WJ-1402] Revamp Wikicomma import script
I've updated the Wikicomma importer system. It now does something very different, it intakes data into an intermediate SQLite database, since this allows me to handle the various foreign key and consistency issues that come up when dealing with scraped data. It also allows partial transformation to Wikijump data standards, such as uploading files to S3 under their hashes, storing text hashes, etc.