etcd-io
boltdb write data by os.file.writeAt, why not use mmap() to write data into db ？
I find boltdb only use mmap() to read data, but you do not use mmap() to write data . you just use os.file.writeAt() to write data into db. The code I find as follow: db.ops.writeAt = db.file.WriteAt
why not use mmap()? It can write data faster.
Not exactly sure what you mean. But if you think there can be some performance gains, can you try to modify the code and do some benchmark to show the result first?
ok， I will manage to find time to modify some code and benchmark .
Thank you for your prompt reply.
If you directly update the mmaped pages, how do you guarantee the OS kernel flush the dirty pages to disk as you want?
If you directly update the mmaped pages, how do you guarantee the OS kernel flush the dirty pages to disk as you want?
@louchenyao, there is 'msync' function for that.
It should be possible to simply copy the pages onto the existing memory mapping, I tried the following but it wasn't sufficient to get it working. Someone more familiar with the code might be able to figure out how to handle things like locking and truncation:
db.ops.writeAt = func(b []byte, off int64) (n int, err error) {
copy(db.data[off:], b)
return len(b), nil
}
as mentioned by @Lawliet-Chan this could allow significantly faster writes, particularly when NoSync is enabled as the pages are just written to RAM and left up to the kernel to decide how/when to flush them to disk.
note: the 'protection bits' used when creating the mapping will need to be changed to
syscall.PROT_READ|syscall.PROT_WRITE
