FerretDB icon indicating copy to clipboard operation
FerretDB copied to clipboard

Make maximum documents size configurable

Open Gowee opened this issue 2 months ago • 4 comments

What should be done?

It appears that FerretDB inherit the 16MiB document size limits from MongoDB. It is not pretty reasonable nowadays.

I think we can have an option to tweak the default limit.

FerretDB version

v1.21.0

Backend

PostgreSQL 12.7

Steps to reproduce

mongoimport a document larger than 16MiB to FerretDB. Then mongoimport exits with the error Failed: an inserted document is too large.

Gowee avatar Apr 12 '24 04:04 Gowee

Let's provide a testing CLI flag for that

AlekSi avatar May 17 '24 19:05 AlekSi

The error appears to come from the driver. See https://github.com/mongodb/mongo-go-driver/blob/v1/x/mongo/driver/batches.go#L17. So the request didn't reached FerretDB as the driver returned error before that. Let's see what we can do before we add testing CLI flag.

chilagrow avatar May 20 '24 02:05 chilagrow

So the request didn't reached FerretDB as the driver returned error before that.

It seems that MaxDocumentSize is a configurable value indicated by the server.

https://github.com/mongodb/mongo-go-driver/blob/345ea9574e28732ca4f9d7d3bb9c103c897a65b8/x/mongo/driver/operation.go#L713

Gowee avatar May 20 '24 05:05 Gowee

That MaxDocumentSize is internal representation of server and not configurable for a driver. See information about x package https://github.com/mongodb/mongo-go-driver/tree/345ea9574e28732ca4f9d7d3bb9c103c897a65b8/x

Update: I don't find implementation within FerretDB restricting maximum document size. And drivers are out of our control unfortunately.

chilagrow avatar May 20 '24 08:05 chilagrow

In fact, this is indeed configurable 🙈 The value that sets driver was the response from isMaster command's maxBsonObjectSize field.

chilagrow avatar May 22 '24 01:05 chilagrow