Value of mdb_mapsize in lmdb-safe.cc

[maxwong]

Hi there, thanks for the great job for this neat project. But I have a question in lmdb-safe.cc, line 30 - 31:

  if(mdb_env_set_mapsize(d_env, 16ULL*4096*244140ULL)) // 4GB
    throw std::runtime_error("setting map size");

But 16ULL * 4096 * 244140ULL != 4GB (4 * 1024 * 1024 * 1024?). Is it a typo or a purposed feature? Not sure about it.

And it will be great if users can set some configs when getting the env, especially those can only be set after calling mdb_env_create() but before mdb_env_open(), (e.g. maxdbs). Is there any plan for this? Maybe something like wrapping those configs in a class and pass it when calling getMdbEnv().

Thanks.

[horazont]

Having this a constant is indeed not great (for starters, the default constant isn’t going to work on 32 bit systems at all).

I’m not sure what a good way of setting it would be though, given that the Environment is essentially a (per file descriptor) singleton:

• Do all calls need to set a (the same) size?
• Would a differing size be an error or a resize request?
• What if a resize request happens while the environment is still open (we’d have to halt all transactions in some way)?

I don’t have necessarily good answers. I have a proposal, but it’d be hard to implement and get right:

• Opening an environment without argument: re-use the size of the (existing) environment
• With size
  • If not open: open and resize
  • If already open and size differs: block until all transactions have finished; block new transactions from starting. Resize the environment. Allow transactions to continue.

Would that work? Would it make sense?

[maxwong]

Sorry for the late reply. Auto resize looks good. But I also suggest giving an option to users, allowing them to disable auto resize.

Error handling strategies may differ in different libraries / systems. It will be difficult to debug if these strategies are different. So I personally prefer more manual control in the storage engine layer. If there is an existing db being open with a different dbsize, throwing an exception or returning an error code (but not core dump) will be good. The application can decide what to do with the error.