Michal Guzek
Michal Guzek
@samurdhikaru could you help advise on the plugin question?
The presence of -1 in the output dimension indicates a dynamic shape. The actual dimension value is determined at inference time, based on the input data and model operations. If...
How about this example? (EDIT: Sorry, the "try to run inference with actual input data" might have been confusing without an example) ``` // Create execution context IExecutionContext* context =...
Are you able to get the true output dimensions of the model after this sequence of calls? ``` Dims inputDims = Dims4(1, 3, 832, 1440); // or whatever your actual...
Apologies for the late reply, there were company's holidays last week. TRT may also need certain optimizations enabled to fully resolve dynamic shapes, you can enable that by using optimization...
Sorry, I had to do more digging because wasn't familiar with that part of the codebase. To unblock you for now, you can use `getMaxOutputSize` to query the max possible...
@demuxin , I believe for now you will have to use `getMaxOutputSize`. If the memory allocation turned out to be too large, let me know and I can CC someone...
Can you paste your full standalone C++ snippet or at least the part where you build the engine and query `getMaxOutputSize()`? I'll instance an internal bug then for someone to...
@demuxin I also got `24064` bytes on the output but just realized that the output type is `float32` and so `1000 * 6 * 4 (sizeof(float32)) = 24000` as expected?
This is an upper bound like I said, so feel free to divide it by `2` when using half precision.