TensorRT
TensorRT copied to clipboard
✨[Feature] Add inputs as metadata to exported programs/graphmodules to avoid passing them to save/compile APIs
Is your feature request related to a problem? Please describe. Our current workflow
ep = torch.export.export(model, (inputs,))
trt_gm = torch_tensorrt.dynamo.compile(ep, inputs=[inputs])
torch_tensorrt.save(trt_gm, "trt.ep", inputs=[inputs])
Desired workflow:
ep = torch.export.export(model, (inputs,))
trt_gm = torch_tensorrt.dynamo.compile(ep)
torch_tensorrt.save(trt_gm, "trt.ep")
In theory, we don't need to pass inputs everytime in this workflow. Consider adding them as metadata to ep or trt_gm in the above example.