wikijump icon indicating copy to clipboard operation
wikijump copied to clipboard

[WJ-1402] Revamp Wikicomma import script

Open emmiegit opened this issue 7 months ago • 0 comments

I've updated the Wikicomma importer system. It now does something very different, it intakes data into an intermediate SQLite database, since this allows me to handle the various foreign key and consistency issues that come up when dealing with scraped data. It also allows partial transformation to Wikijump data standards, such as uploading files to S3 under their hashes, storing text hashes, etc.

emmiegit avatar Jul 05 '24 03:07 emmiegit