yorkie Snapshot overflow

Snapshot overflow

Open hackerwins opened this issue 2 years ago • 2 comments

What happened:

If a document's snapshot exceeds 16 MB, the following error occurs:

2021-11-04T06:17:47.355Z	ERROR	an inserted document is too large
github.com/yorkie-team/yorkie/yorkie/backend/db/mongo.(*Client).CreateSnapshotInfo
	/app/yorkie/backend/db/mongo/client.go:371
github.com/yorkie-team/yorkie/yorkie/packs.storeSnapshot
	/app/yorkie/packs/packs.go:418
github.com/yorkie-team/yorkie/yorkie/packs.PushPull.func1
	/app/yorkie/packs/packs.go:150
github.com/yorkie-team/yorkie/yorkie/backend.(*Backend).AttachGoroutine.func1
	/app/yorkie/backend/backend.go:164
2021-11-04T06:17:47.366Z	ERROR	an inserted document is too large
github.com/yorkie-team/yorkie/yorkie/packs.PushPull.func1
	/app/yorkie/packs/packs.go:155
github.com/yorkie-team/yorkie/yorkie/backend.(*Backend).AttachGoroutine.func1
	/app/yorkie/backend/backend.go:164

This is because there is a limit on the size of documents in MongoDB.

To store documents larger than the maximum size, MongoDB provides the GridFS API. See mongofiles and the documentation for your driver for more information about GridFS.

It might be a good idea to check if we can use GridFS.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment: