thetadata-python icon indicating copy to clipboard operation
thetadata-python copied to clipboard

changes in stream parsing

Open mukrrmsson opened this issue 2 years ago • 0 comments

Instead of asking the socket each time, byte amount of a contract, quote, trade etc., we ask a certain buffer size (e.g. 4KB) and wait until socket delivers it. Upon reception, the whole byte buffer is parsed. I find this faster than the original method. However, there are some caveats:

  1. Socket will wait until there is enough data (i.e. buffer size), this can introduce delays into data delivery into callback if the buffer size is too high compared to expected delivered number of messages as the socket will wait.
  2. I noticed that on my Windows 11 machine, if buffer size is too small, sometimes ethernet crashes. Not sure if this is OS or device or Theta Terminal.
  3. Therefore a good balance needs to be found where we do not get throttled by OS and also dont get delayed due to longer waiting times at socket until buffer size is filled.

mukrrmsson avatar Oct 24 '23 11:10 mukrrmsson