store
store copied to clipboard
Add routines for peeking/poking from/to file
Currently Peek
/Poke
operations can be run on strict ByteString
s or Ptr
s. I suggest adding routines that run them given a FilePath
, and implement it using mmap
, which avoids copying and should have good performance.
Good idea! Could probably use https://hackage.haskell.org/package/mmap-0.5.9/docs/System-IO-MMap.html very directly. Not sure if I want to add mmap as a direct dep, could be worth suggesting in the documentation. The docs make it sound like using this approach for writing the file could be dicey.
How about just decode <$> mmapFileByteString fp Nothing
?
That would be fine, but works only for peeking (mmapFileByteString
maps file in ReadOnly
mode). We should use mmapFilePtr
and play with raw Ptr
s. I'll write a benchmark to see if it is truly beneficial for using mmap
when I got spare time.
@mgsloan Sorry for the late reply. I implemented the operations in a separate package here: https://github.com/TerrorJack/store-file (not on Hackage yet). It can run Peek
/Poke
operations given a FilePath
, and has mmap
-based/ByteString
-based flavors.
I also implemented a simple test-suite and a benchmark, reading/writing HashMap ByteString ByteString
. Surprisingly, benchmark shows mmap
-based implementation actually a little bit slower with large examples. Any idea why?
Cool, good stuff! Perhaps the performance difference is because there are no {-# INLINE #-}
pragmas?