Persisting State: Documentation?

[canadaduane]

As per your comment, it would be great to learn more about persisting state in conjunction with y-websocket. Thanks!

_{Huly®: YJS-533}

[dmonad]

Hey @canadaduane , I'm currently updating y-redis and y-leveldb. I'll also restructure the API that adds persistence to y-websocket. Until now, I expected people to clone the y-websocket server, but I think it makes sense to at least allow users to share persistence modules properly.

[canadaduane]

That would be fantastic! Thank you.

[renieravin]

Hey @dmonad , any luck on this front? Would love to implement persistence in Yjs 13!

[dmonad]

@renieravin Sure, I'd love to get a contribution on y-leveldb.

If you want to get started, it would be best to start a new project based on y-indexeddb and rewrite it based on leveldb. I recommend to use [leveldown]https://github.com/Level/leveldown) unless there is a good alternative.

[dmonad]

Actually, @renieravin I just might have found a sponsor for the leveldb adapter. I'll let you know if it works out :)

[anilgulecha]

@dmonad was any update made on this persistence feature? The README does talk about:

PORT=1234 YPERSISTENCE=./dbDir node ./node_modules/y-websocket/bin/server.js

So is this available for use, or a WIP?

[mikkorantalainen]

Any new info about persisting state? I'd prefer pushing and pulling the persisting state to/from Postgres database so any pointers about that would be great.

[AdventureBeard]

@mikkorantalainen, you might be able to rig up a connection to Postgres by leveraging y-leveldb + level + SQLdown, though I'd be careful with SQLdown, as it looks pretty old and opinionated. You might use SQLdown as inspiration for fulfilling the level interface in a way that best suits your application's needs. Haven't tried level + SQLdown myself yet, but probably will pretty soon. Will report back if so.

[rebolyte]

Any update here? Being able to drop in y-redis here would be 🔥

[dmonad]

Hi @rebolyte

No update.

In the limited time I have for this project (very little funding), I'm working on the new Yjs documentation site.

y-leveldb is stable now and could also be used over the different databases that levelup supports.

[jnbarlow]

@dmonad The sample project has :

// Super hacky way to provide a initial value from the client, if
		// you plan to use y-websocket in prod you probably should provide the
		// initial state from the server.

but I can't find docs and/or figure out how to send the state up on the initial connection.
I've got this far:

utils.setPersistence({
	bindState: async (docName, yDoc) => {
		log.debug('bindState:', docName);
		//load from db and seed?
		const document = yDoc.getArray();
		console.log(document.toJSON());
		document.insert(0, [{ type: 'paragraph', children: [{ text: 'Hello world from server!' }] }]);
		console.log(document.toJSON());

		yDoc.on('update', update => {
			//do some crazy saving here.
			const doc = utils.getYDoc(docName).getArray('content').toJSON();
			log.debug(docName);
			log.debug(JSON.stringify(doc, null, 2));
		});
	},
	writeState: async (docName, yDoc) => {
		log.debug('writeState:', docName);
		//this fires on connection close
	}
});

and document has the initial state, but I just for the life of me can't figure out how to push that to the front end.

[hanspagel]

Just wanted to chime in and say that we’re working on a wrapper on top of y-websocket, y-redis, and y-leveldb (in collaboration with Kevin) called hocuspocus. If you struggle to put together all the building blocks, it’s probably worth a look.

It’s only available for GitHub sponsors for now, but we’ll publish it as a MIT-licensed package in the next months. If you can’t afford sponsoring our efforts, reach out to [email protected] and I’ll invite you manually.

[bxff]

Any updates on this? Should we use utils.setPersistence to save persistences like y-mongodb-provider or use levelup to store the data on other databases?