bqskit
bqskit copied to clipboard
Runtime updates
Improves the runtime by addressing some shortcomings that cause issues in rare situations and implementing features that provide quality of life improvements:
- Splits the single-threaded worker into two threads: one thread does work, while one consumes incoming messages. This prevents the rx buffer from filling up and blocking other parts of the system while the worker is working on a long job.
- Added read receipts to the waiting messages to create a "happens-before" relationship with certain messages. This ensures eventual consistency with idle worker tracking through the system and avoids a previously allowed race condition. Moreover, the manager can now always adjust their idle count to an accurate amount.
- Workers now send messages instantly rather than waiting until their current job has finished or enters an awaited state. This allows for child tasks to start quicker and log messages to appear instantly.
- Improved Serialization
- Circuit serialization now uses dill internally (even if called with pickle) for gates that cannot be pickled normally
- Circuit serialization uses pickle (even if called with dill) to pickle gates that can be pickled normally
- Workflow serialization now uses dill internally, allowing lambdas to be passed as pass arguments (and more)
- Additionally, workflows using dill now allows passes defined in
__main__
to be used without exporting to another file - RuntimeTasks only deserialize fnargs at destination now rather than at every node in the runtime
- RuntimeTasks use dill to serialize fnargs allowing lambdas to be passed to children tasks (and more)
- Compilers export their sys path to the runtime, allowing normally unreachable files to be found, depending on file system similarities
- Fixes: #181
- Fixes: Attached servers and workers importing numpy before setting BLAS threads to 1
- Fixes: Workflow copy constructor not setting all necessary variables
- Address: #211
TODO:
- [ ] Evaluate performance hit with using dill
- [x] Allow BLAS thread settings to be configurable on server launch
- [x] Improve logging capabilities of workers
- [x] Fix #213
- [x] Fix #198
- [x] Consider #226