pgcontents icon indicating copy to clipboard operation
pgcontents copied to clipboard

[documentation] migration to/from filesystem

Open snakescott opened this issue 7 years ago • 2 comments

Moving state out of the system running jupyter and into a DB sounds awesome!

If I have lots of notebooks on disk, is there a clean way to import them into a pgcontents backend? Similarly, if I want to rollback from pgcontents to the file system, is there a clean way to export?

snakescott avatar Aug 06 '17 07:08 snakescott

@snakescott sorry for the long delay in replying to this.

We actually used to have a utility that did basically what you're describing for checkpoints, but I ended up removing it for lack of use. More recently, @nathanwolfe did work in (https://github.com/quantopian/pgcontents/pull/30) to add tools in pgcontents.query to produce a generator over all the files in the database, handling decryption and taking min/max timestamps to support resuming if the process crashes. You could build a "sync user to local directory" tool on top of that system relatively straightforwardly.

ssanderson avatar Oct 12 '17 16:10 ssanderson

Is this still an issue?

dmichalowicz avatar Feb 11 '20 17:02 dmichalowicz