systems icon indicating copy to clipboard operation
systems copied to clipboard

Use GPU tensors in Triton ensemble operators

Open jperez999 opened this issue 2 years ago • 0 comments

  • [ ] Use DLpack to leverage GPU memory between triton models in ensemble
  • [ ] Upgrade numpy in containers and see if dlpack works with Triton tensors
  • [ ] Try to build a repro of transferring cupy tensors to Triton with dlpack (re: issue with contiguous arrays)

jperez999 avatar Mar 14 '22 14:03 jperez999