space_packet_parser icon indicating copy to clipboard operation
space_packet_parser copied to clipboard

ENH: Change parsing logic to bytes manipulation directly

Open greglucas opened this issue 1 year ago • 0 comments
trafficstars

Speed up parsing by removing bitstring reads

Avoid bitstring and do the bitwise reading and manipulations ourselves with bitwise logic. This greatly increases the performance of the parser by ~3-5x in the read routine. This is sort of the MVP for speed-ups by just implementing the routines we need and trying to replicate bitstring. We could do better by refactoring the _get_format_string() logic and avoiding any string matching/parsing/splitting too.

I did not fully remove bitstring at this point since there is a lot of logic surrounding it in parser.py. We could make it an optional dependency, but it would be nice to get rid of the legacy_parse_packet() method if we go that route. Happy to add that deprecation path here if it'd be useful, but probably a major version bump then.

Profiling

main image

this branch image

Checklist

  • [x] Changes are fully implemented without dangling issues or TODO items
  • [ ] Deprecated/superseded code is removed or marked with deprecation warning
  • [ ] Current dependencies have been properly specified and old dependencies removed
  • [n/a] New code/functionality has accompanying tests and any old tests have been updated to match any new assumptions
  • [ ] The changelog.md has been updated

greglucas avatar Jul 30 '24 12:07 greglucas