Burhan
Burhan
I don't believe this is required for the PR, but adding here in a comment for potential future reference. Script for benchmark tests ```py import time import subprocess from pathlib...
@glenn-jocher and @Laughing-q TensorRT INT8 export + calibration is ready for review! Lots of changes here, but hoping it'll be close to a ready state. TensorRT `INT8` inference is ~25%...
Found an issue with `classify` datasets and working on a solution now. What I'm attempting with my solution might make it possible to revert the inference preprocessing step, but I'm...
Made several attempts to not make any changes to pre-processing, but all alternative `Dataloaders` threw errors and were not able to load data for calibration.
Finally, success! Some of the earlier issues seem to be related to #10424 which is why I moved away from using `ClassificationDataset` initially. The final mini-benchmark tests are running now...
@glenn-jocher PR is ready. I was able to get `YOLODataset` working with `batch` when handled by `InfiniteDataloader` and that means that inference preprocessing has been restored to the original design....
@glenn-jocher yes we could hard code the quantization calibration algorithm for now. I think I'll run a quick test to see if there's any detectable difference in the calibration results...
From the [TensorRT docs](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#enable_int8_c) > Calibration batch size can also affect the truncation error for IInt8EntropyCalibrator2 and IInt8EntropyCalibrator. For example, calibrating using multiple small batches of calibration data may result...
TensorRT algorithm testing found that only `ENTROPY_CALIBRATION_2` and `MINMAX_CALIBRATION` seem to work, with `ENTROPY_CALIBRATION_2` having **significantly** better performance with COCO128 for validation | Calibration | Exported | Images | Instances...
Running `test_export_engine_matrix` locally while I grab some lunch. I thought it made sense to keep with `test_cuda.py` since it _has_ to run on GPU.