Po-Wei (Vincent)
Po-Wei (Vincent)
after converting onnx fp32 to int8 engine with custom calibration, the engine layers still show fp32
@jinhonglu sharing via a google drive link works as well
Closing as no updates from requester for +10 days. Feel free to reopen when you have some bandwidth to work on it!
@nv-guomingz can you please take a look at this PR?
Closing based on https://github.com/NVIDIA/TensorRT-LLM/pull/428#issuecomment-2949212926
Closing as no activities from requester for +10 days. Feel free to reopen when you have bandwidth to work on this!
Closing based on https://github.com/NVIDIA/TensorRT-LLM/issues/2964#issuecomment-2754585600. Feel free to reopen 👍
Closing due to inactive. Please feel free to reopen!
TRT will not fuse ops or remove dead nodes that produces graph outputs so marking them with `--trt-outputs mark all` will stop the fusion. For other questions let me get...
You could try to use this code to mark all outputs to onnx first and then build the TRT network. This is also how polygraphy `--mark-outputs` does [in the background](https://github.com/NVIDIA/TensorRT/blob/64e56ab6cbe95c8220d80bc7ff10ccf61935759a/tools/Polygraphy/polygraphy/tools/args/util/util.py#L111-L125)....
@akhilg-nv The labels are queried by the time the action was triggered. The labels are inconsistent to the labels we have in the main repo because the issue was raised...