pgcontents
pgcontents copied to clipboard
[documentation] migration to/from filesystem
Moving state out of the system running jupyter and into a DB sounds awesome!
If I have lots of notebooks on disk, is there a clean way to import them into a pgcontents backend? Similarly, if I want to rollback from pgcontents to the file system, is there a clean way to export?
@snakescott sorry for the long delay in replying to this.
We actually used to have a utility that did basically what you're describing for checkpoints, but I ended up removing it for lack of use. More recently, @nathanwolfe did work in (https://github.com/quantopian/pgcontents/pull/30) to add tools in pgcontents.query
to produce a generator over all the files in the database, handling decryption and taking min/max timestamps to support resuming if the process crashes. You could build a "sync user to local directory" tool on top of that system relatively straightforwardly.
Is this still an issue?