toxiproxy
toxiproxy copied to clipboard
Update ChanReader / ChanWriter for use inside toxics
While working on the bidirectional toxics PRs I discovered some problems with how ChanReader worked when used ephemerally inside a toxic.
These turned out to be non-trivial to fix, with the end result being this PR.
The problem with ChanReader is that in some cases it may buffer data. This means if you ever stop using the reader and switch to another, there's a possibility of losing data.
Further problems arise when ChanReader is read using other readers such as bufio.Reader as required by functions such as http.ReadRequest().
These problems are solved in 3 ways:
- Use a non-ephemeral reader object on the
ToxicStubinstead of creating it every timePipe()is called. - Create a
TransactionalReaderthat can be rolled back to specific points so thatbufio.Readercan effectively be un-read. - Allow all buffers inside the reader to be explicitly flushed at any point so that no data is lost.
Example TransactionalReader usage:
// Input is "hello world"
txnReader := stream.NewTransactionalReader(input)
{ // This section is ephemeral (and would be similar to usage in `Pipe()`)
bufReader := bufio.NewReader(txnReader)
msg, _ := bufReader.ReadString(' ')
// msg == "hello "
// bufReader has "world" buffered (bufReader.Buffered() == 5)
// Mark everything up to "hello " as read.
txnReader.Checkpoint(-bufReader.Buffered())
// Rollback txnReader to the checkpoint, we never read "world"
txnReader.Rollback()
// bufReader can now be thrown away
}
// Any further reads from txnReader will start from the checkpoint
buf := make([]byte, 5)
txnReader.Read(buf)
// buf == "world"
Actual usage can be found in my WIP redis toxic here: https://github.com/Shopify/toxiproxy/blob/redis-wip/toxics/redis.go
The documentation for this is pretty tricky to write without actual code examples, so I'll likely leave the detailed docs until we have at least 1 protocol-aware toxic to point at.
@Sirupsen @jpittis / anyone else interested in Golang
Went through this again and cleaned some things up now that it's not fresh in my mind. Should be good for a review @sirupsen