TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard

✨[Feature] Add inputs as metadata to exported programs/graphmodules to avoid passing them to save/compile APIs

Open peri044 opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Our current workflow

ep = torch.export.export(model, (inputs,))
trt_gm = torch_tensorrt.dynamo.compile(ep, inputs=[inputs])
torch_tensorrt.save(trt_gm, "trt.ep", inputs=[inputs])

Desired workflow:

ep = torch.export.export(model, (inputs,))
trt_gm = torch_tensorrt.dynamo.compile(ep)
torch_tensorrt.save(trt_gm, "trt.ep")

In theory, we don't need to pass inputs everytime in this workflow. Consider adding them as metadata to ep or trt_gm in the above example.

peri044 avatar Mar 20 '24 18:03 peri044