sbe-python icon indicating copy to clipboard operation
sbe-python copied to clipboard

Fix set type bit order

Open threewholefish opened this issue 3 years ago • 2 comments

Python bistrings are processed MSB-first, but SBE set types are processed LSB first. Therefore, we need to reverse the encoding/decoding order as parsed.

This may need to be tested for sets which are larger than 1 byte

threewholefish avatar May 13 '22 08:05 threewholefish

This changes how the parser works. Since I and other users have been using this for real world data and apparently those worked, can you provide some example data of why this change is needed?

kizzx2 avatar May 15 '22 15:05 kizzx2

Take this set from the CME MDP3 spec:

<set name="MatchEventIndicator" encodingType="uInt8">                                                  
    <choice name="LastTradeMsg" description="1=Last trade message for the event, 0=Not last">0</choice>
    <choice name="LastVolumeMsg" description="1=Last electronic volume message, 0=Not last">1</choice>
    <choice name="LastQuoteMsg" description="1=Last real quote message, 0=Not last">2</choice>
    <choice name="LastStatsMsg" description="1=Last statistics message, 0=Not last">3</choice>
    <choice name="LastImpliedMsg" description="1=Last implied quote message, 0=Not last">4</choice>
    <choice name="RecoveryMsg" description="1=Message is sent during  recovery process">5</choice>
    <choice name="Reserved" description="0=Reserved for future use">6</choice>
    <choice name="EndOfEvent" description="1=Last message for the event, 0=Not last">7</choice>
</set>

The raw value 0x1 should correspond to a decoded value of ["LastTradeMsg"], but it is instead decoded as ["EndOfEvent"]. Encoding is consistent. As best I can tell, this is because the SBE spec states that the bit indices start from the LSB, but bitstring.Bits and bitstring.Bitarray start from the MSB.

threewholefish avatar May 15 '22 17:05 threewholefish