store icon indicating copy to clipboard operation
store copied to clipboard

Add routines for peeking/poking from/to file

Open TerrorJack opened this issue 8 years ago • 4 comments

Currently Peek/Poke operations can be run on strict ByteStrings or Ptrs. I suggest adding routines that run them given a FilePath, and implement it using mmap, which avoids copying and should have good performance.

TerrorJack avatar Jul 27 '16 04:07 TerrorJack

Good idea! Could probably use https://hackage.haskell.org/package/mmap-0.5.9/docs/System-IO-MMap.html very directly. Not sure if I want to add mmap as a direct dep, could be worth suggesting in the documentation. The docs make it sound like using this approach for writing the file could be dicey.

How about just decode <$> mmapFileByteString fp Nothing ?

mgsloan avatar Jul 28 '16 22:07 mgsloan

That would be fine, but works only for peeking (mmapFileByteString maps file in ReadOnly mode). We should use mmapFilePtr and play with raw Ptrs. I'll write a benchmark to see if it is truly beneficial for using mmap when I got spare time.

TerrorJack avatar Jul 28 '16 22:07 TerrorJack

@mgsloan Sorry for the late reply. I implemented the operations in a separate package here: https://github.com/TerrorJack/store-file (not on Hackage yet). It can run Peek/Poke operations given a FilePath, and has mmap-based/ByteString-based flavors.

I also implemented a simple test-suite and a benchmark, reading/writing HashMap ByteString ByteString. Surprisingly, benchmark shows mmap-based implementation actually a little bit slower with large examples. Any idea why?

TerrorJack avatar Aug 06 '16 05:08 TerrorJack

Cool, good stuff! Perhaps the performance difference is because there are no {-# INLINE #-} pragmas?

mgsloan avatar Aug 08 '16 20:08 mgsloan