bfile
bfile copied to clipboard
A buffer pool file I/O library for Go
bfile
A buffer pool file I/O library for Go.
The purpose of the library is to provide I/O mechanisms for copying data to and from a buffer in user space and maintain complete control over how and when it transfers pages.
This is an alternative to using mmap
on large DBMS like files.
Install
go get github.com/tidwall/bfile
Usage
Use the NewPager
function to create a new Pager
, which includes three
functions: ReadAt
, WriteAt
, and Flush
.
The Pager reading and writing works like an os.File
, but with an
automatically maintained pool of buffered pages.
// Open a file for read/write
f, err := os.OpenFile("bigfile.dat", os.O_RDWR, 0)
// Create a new Pager for accessing the data in opened file.
p := bfile.NewPager(f)
// Read some data at a specific offset.
data := make([]byte, 50)
n, err := p.ReadAt(data, 827364)
// Write some data at the same offset.
n, err := p.WriteAt([]byte("hello"), c)
// Flush unwritten data to the underlying os.File
err = p.Flush()
There's also a Stream
object for sequentially reading and writing data, which
includes the three functions: Read
, Write
, and Flush
.
// Create a new Pager
p := bfile.NewPager(f)
// Create a Stream that is backed by the Pager starting at a specific offset
s := p.Stream(827364)
// Read 50 bytes data at the offset 827364.
data := make([]byte, 50)
n, err := s.Read(data)
// Write the string "hello" at 827414, which is after the previous read.
n, err := s.Write([]byte("hello"))
// Flush unwritten data to the underlying os.File
err = p.Flush()
The default page size is 4096 and the default size of all buffered pages will
not exceed 8 MB.
These defaults can be changed by using the NewPagerSize
function.
All operations are thread-safe.
Contact
Josh Baker @tidwall
License
Source code is available under the MIT License.