butterfly icon indicating copy to clipboard operation
butterfly copied to clipboard

Refactor and update parsing

Open moofMonkey opened this issue 2 years ago • 0 comments

Fixes #7 Fixes #8

  • [x] Fix docker build
  • [x] Fix stringtables parsing
  • [x] Fix deprecation warnings
  • [x] Add support for stringtables using varint_bitcounts
  • [x] Fix fp_PushN
  • [x] Clean up and optimize fieldpath code
  • [x] Implement resource manifest parsing to avoid embedding resources trie in butterfly binary/sources
  • [x] Rework flattened serializers
    • [x] Handle all arrays as if they're dynamic
    • [x] Move property#type to fs_info
    • [x] Avoid storing property paths as strings and introduce new property getter API to reduce RAM usage
    • [x] Construct entities layout inside flattened_serializer instead of using std::vector/std::unordered_map to reduce RAM and CPU usage
  • [ ] Fix m_fGameTime logic - m_fGameTime was removed in favor of tick-based time calculation
  • [ ] Implement proper entities baselines support
  • [ ] Implement stringtables callbacks
  • [ ] Implement entities props callbacks nor specify changed paths in on_entity callback?
  • [ ] Eliminate BUTTERFLY_THREADSAFE
    • [ ] Eliminate global object pools
    • [ ] Replace static std::vector with llvm::SmallVector in entity::parse
  • [ ] Discuss and reimplement JS/Python APIs using event-driven approach
  • [ ] Discuss whether we want to store resource paths as strings or provide API for resource path resolution
  • [ ] Discuss clang-format Would it be fine if I add Google/LLVM-styled clang-format? If so, is SpacesInParentheses: true worth preserving in butterfly code?
  • [ ] Fix Windows builds
  • [ ] Run unit tests against matches in different game patches?
  • [ ] Add support for GitHub CI?
  • [ ] Store statically sized arrays without additional allocations? This might introduce problems if array size is parsed incorrectly, but could theoretically reduce RAM/CPU usage.

Known problems:

  • Possible crash on exit in case of global butterfly::parser due to global object pools heap use-after-free in entity::~entity This affects 03-props, 06-events and 07-combatlog
  • entity_classes class_id should either assert or return UINT32_MAX on lookup failure

Next PRs TODO:

  1. Proper VPK API
  2. KV parsing API

moofMonkey avatar Oct 14 '23 11:10 moofMonkey