DCHECK(kind_ == value_kind_symbolic_) failed

[tsaizhenling]

Description

onnx to trt conversion fails for model with dynamic batch

Environment

TensorRT Version: 8.5.2.2

NVIDIA GPU: xavier nx

CUDA Version:12.2

Operating System: ubuntu 20.04

Python Version (if applicable): 3.12.4

Relevant Files

Model link: https://drive.google.com/file/d/13l9CUXUJOiHfth-ryRFtxuq7vlpm1Kur/view?usp=sharing

Steps To Reproduce

import numpy as np                                                                                                                                                                                                                   
from polygraphy.backend.trt import (                                                                                                                                                                                                 
    CreateConfig,                                                                                                                                                                                                                    
    EngineFromNetwork,                                                                                                                                                                                                               
    NetworkFromOnnxPath,                                                                                                                                                                                                             
    SaveEngine,                                                                                                                                                                                                                      
    TrtRunner,                                                                                                                                                                                                                       
    Profile                                                                                                                                                                                                                          
)   
onnx_model = "parseq_recognizer_fix_dynamicbatch.onnx"

profiles = [
        Profile().add("input", min=(1, 3, 32, 128), opt=(1, 3, 32, 128), max=(1, 3, 32, 128)),
        Profile().add("input", min=(1, 3, 32, 128), opt=(4, 3, 32, 128), max=(10, 3, 32, 128)),
        Profile().add(
            "input", min=(10, 3, 32, 128), opt=(10, 3, 32, 128), max=(10, 3, 32, 128)
        ),
    ]

def main():

    inp_data = np.ones(shape=(1, 3, 32, 128), dtype=np.float32)
    rsess = InferenceSession(onnx_model, 
        providers=["CUDAExecutionProvider"]) 
    pred = rsess.run(None, {"input": inp_data})
    print(pred)

    build_engine = EngineFromNetwork(
        NetworkFromOnnxPath(onnx_model), config=CreateConfig(fp16=False, profiles=profiles)
    )
    build_engine = SaveEngine(build_engine, path="parseq_test.engine")
    with TrtRunner(build_engine) as runner:
        outputs = runner.infer(feed_dict={"input": inp_data})
        print(outputs)

if __name__ == "__main__":
    main()

error:

[W] onnx2trt_utils.cpp:375: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.                                                                 
[W] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped                                                                                                                                               
[W] Tensor DataType is determined at build time for tensors not marked as input or output.                                                                                                                                           
[I] Configuring with profiles:[                                                                                                                                                                                                      
        Profile 0:                                                                                                                                                                                                                   
            {input [min=(1, 3, 32, 128), opt=(1, 3, 32, 128), max=(1, 3, 32, 128)]},                                                                                                                                                 
        Profile 1:                                                                                                                                                                                                                   
            {input [min=(1, 3, 32, 128), opt=(4, 3, 32, 128), max=(10, 3, 32, 128)]},                                                                                                                                                
        Profile 2:                                                                                                                                                                                                                   
            {input [min=(10, 3, 32, 128), opt=(10, 3, 32, 128), max=(10, 3, 32, 128)]}                                                                                                                                               
    ]                                                                                                                                                                                                                                
[I] Building engine with configuration:                                                                                                                                                                                              
    Flags                  | []                                                                                                                                                                                                      
    Engine Capability      | EngineCapability.DEFAULT                                                                                                                                                                                
    Memory Pools           | [WORKSPACE: 6854.28 MiB]                                                                                                                                                                                
    Tactic Sources         | [CUBLAS, CUBLAS_LT, CUDNN, EDGE_MASK_CONVOLUTIONS, JIT_CONVOLUTIONS]                                                                                                                                    
    Profiling Verbosity    | ProfilingVerbosity.DETAILED                                                                                                                                                                             
    Optimization Profiles  | 3 profile(s)                                                                                                                                                                                            
[W] DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU                                                                                                                              
[W] Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.                                                                                                          
value.h:413: DCHECK(kind_ == value_kind_symbolic_) failed.                                                                                                                                                                           
Aborted (core dumped)