TensorRT
TensorRT copied to clipboard
✨[Feature] Use a R/W Lock to manage building engines for MGMN setups
Is your feature request related to a problem? Please describe.
Describe the solution you'd like
An implementation of the engine cache that is concurrency aware. Should spin lock repeated requests until they are satisfied by one of the workers
Describe alternatives you've considered
Additional context