go-diskqueue

go-diskqueue copied to clipboard

check current file against maxBytesPerFile

3

Fixes #38

mreiferson

Peek multiple

1

I use diskqueue as a staging local queue before pushing to Kafka. I use `sarama` go kafka library for Kafka part. I need to have high reliability of delivery, and...

baryluk

This happens when write/read operations are done concurrently and the queue buffer file is rolled

lootek

.bad files are created when r/w operations happen alternately until queue file is rolled

1

Reproduction as unit test: https://github.com/nsqio/go-diskqueue/pull/37

lootek

Configurable total size limit and delete old data when exceeded

5

Right now new data `Put()` would fail if disk is out of space. It would be great to have an option to 1. limit the total disk space consumed by...

leonzz

diskqueue: support a callback when the disk queue is empty

3

This supports nsqio/nsq#1305

jehiah

Files marked .bad after "Too many open files" read error

3

The current diskqueue implementation does not discriminate between different types of errors returned by `os.OpenFile()`. We experienced a problem in production where an nsqd host ran out of file descriptors...

rdingwall

Close() may cause data lose

1

```ruby func (d *diskQueue) exit(deleted bool) error { d.Lock() defer d.Unlock() d.exitFlag = 1 if deleted { d.logf(INFO, "DISKQUEUE(%s): deleting", d.name) } else { d.logf(INFO, "DISKQUEUE(%s): closing", d.name) } close(d.exitChan)...

trash-boy

Consistent EOF error when reading immediately after write rolls files

1

Consistent EOF error happens if the reading position is keeping up with the writing position and if the write rolls a file, then read will throw an EOF error since...

jonasfreyr

Eof edge case fix

3

This is to address issue https://github.com/nsqio/go-diskqueue/issues/42

jonasfreyr