Is it possible to change the resolution when using TensorRT?

[KeitoTakaishi]

I tried changing the resolution while using TensorRT, but the generation didn't work well. I also rebuilt ONNX and engines, but I couldn't change the resolution. How do I solve this problem?

[ccarmatic]

Hello, I have a similar problem, trying to do this

stream = StreamDiffusion(
    pipe,
    t_index_list=[32, 45],
    torch_dtype=TORCH_DTYPE,
    width=1920,
    height=1080,
    do_add_noise=False,
)

from streamdiffusion.acceleration.tensorrt import accelerate_with_tensorrt

stream = accelerate_with_tensorrt(
    stream, "engines", max_batch_size=2,
)

results in https://github.com/cumulo-autumn/StreamDiffusion/issues/152 ...

if TensorRT is turned on, and if width and height are any value other than 512, it will give an error with nvinfer1:

[E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2,4,67,120] for bindings[0] exceed min ~ max range at index 2, maximum dimension in profile is 64, minimum dimension in profile is 64, but supplied dimension is 67.

[potesd]

I found that I had to hardcode changes to the resolution variables in the Config.json engine name, the wrapper.py width and height, the pipeline.py width and height, and the builder.py (inside of accelerator tensorrt) width and height. With all these values adjusted and your engine named correctly, you can build up to 1024x1024 resolution tensorrt engines at any resolution divisible by 64.

[ccarmatic]

I found that I had to hardcode changes to the resolution variables in the Config.json engine name, the wrapper.py width and height, the pipeline.py width and height, and the builder.py (inside of accelerator tensorrt) width and height. With all these values adjusted and your engine named correctly, you can build up to 1024x1024 resolution tensorrt engines at any resolution divisible by 64.

where can I find this Config.json file?

[potesd]

I found that I had to hardcode changes to the resolution variables in the Config.json engine name, the wrapper.py width and height, the pipeline.py width and height, and the builder.py (inside of accelerator tensorrt) width and height. With all these values adjusted and your engine named correctly, you can build up to 1024x1024 resolution tensorrt engines at any resolution divisible by 64.

where can I find this Config.json file?

my bad, I'm using OlegChomp's StreamDiffusion-NDI for my purposes and it has a config that allows for easy swapping of models and some settings.

[ccarmatic]

my bad, I'm using OlegChomp's StreamDiffusion-NDI for my purposes and it has a config that allows for easy swapping of models and some settings.

I figured it out, you need to pass the image resolution size as a nested dict in accelerate_with_tensorrt , like this:

resolutiondict = {'engine_build_options' : {'opt_image_height': HEIGHT, 'opt_image_width': WIDTH}}
stream = accelerate_with_tensorrt(
    stream, "engines", max_batch_size=BATCH_SIZE,engine_build_options=resolutiondict
)

unfortunately , the 16gb of RAM on my RTX4080 can only support quite a low resolution, and the CUDA 'system fallback policy' (which uses system RAM when there is not enough GPU RAM) isn't going to help here because the engine building script specifically queries the GPU RAM value

[sh308178685]

my bad, I'm using OlegChomp's StreamDiffusion-NDI for my purposes and it has a config that allows for easy swapping of models and some settings.

I figured it out, you need to pass the image resolution size as a nested dict in accelerate_with_tensorrt , like this:

resolutiondict = {'engine_build_options' : {'opt_image_height': HEIGHT, 'opt_image_width': WIDTH}}
stream = accelerate_with_tensorrt(
    stream, "engines", max_batch_size=BATCH_SIZE,engine_build_options=resolutiondict
)

unfortunately , the 16gb of RAM on my RTX4080 can only support quite a low resolution, and the CUDA 'system fallback policy' (which uses system RAM when there is not enough GPU RAM) isn't going to help here because the engine building script specifically queries the GPU RAM value

I do like this ,still can not get the error [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2,4,67,120] for bindings[0] exceed min ~ max range at index 2, maximum dimension in profile is 64, minimum dimension in profile is 64, but supplied dimension is 67.

[ccarmatic]

my bad, I'm using OlegChomp's StreamDiffusion-NDI for my purposes and it has a config that allows for easy swapping of models and some settings.

I figured it out, you need to pass the image resolution size as a nested dict in accelerate_with_tensorrt , like this:

resolutiondict = {'engine_build_options' : {'opt_image_height': HEIGHT, 'opt_image_width': WIDTH}}
stream = accelerate_with_tensorrt(
    stream, "engines", max_batch_size=BATCH_SIZE,engine_build_options=resolutiondict
)

unfortunately , the 16gb of RAM on my RTX4080 can only support quite a low resolution, and the CUDA 'system fallback policy' (which uses system RAM when there is not enough GPU RAM) isn't going to help here because the engine building script specifically queries the GPU RAM value

I do like this ,still can not get the error [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2,4,67,120] for bindings[0] exceed min ~ max range at index 2, maximum dimension in profile is 64, minimum dimension in profile is 64, but supplied dimension is 67.

delete the 'engines' folder, and make it generate a new engine based on the settings in accelerate_with_tensorrt

[sh308178685]

my bad, I'm using OlegChomp's StreamDiffusion-NDI for my purposes and it has a config that allows for easy swapping of models and some settings.

I figured it out, you need to pass the image resolution size as a nested dict in accelerate_with_tensorrt , like this:

resolutiondict = {'engine_build_options' : {'opt_image_height': HEIGHT, 'opt_image_width': WIDTH}}
stream = accelerate_with_tensorrt(
    stream, "engines", max_batch_size=BATCH_SIZE,engine_build_options=resolutiondict
)

unfortunately , the 16gb of RAM on my RTX4080 can only support quite a low resolution, and the CUDA 'system fallback policy' (which uses system RAM when there is not enough GPU RAM) isn't going to help here because the engine building script specifically queries the GPU RAM value

I do like this ,still can not get the error [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2,4,67,120] for bindings[0] exceed min ~ max range at index 2, maximum dimension in profile is 64, minimum dimension in profile is 64, but supplied dimension is 67.

delete the 'engines' folder, and make it generate a new engine based on the settings in accelerate_with_tensorrt

thanks I delete the engines folder,it build success,but still error as the image show

[ccarmatic]

thanks I delete the engines folder,it build success,but still error as the image show

what is your BATCH_SIZE value?

[sh308178685]

thanks I delete the engines folder,it build success,but still error as the image show

what is your BATCH_SIZE value?

you mean the max batch size? I tried 1 and 2 ,both error

[sh308178685]

thanks I delete the engines folder,it build success,but still error as the image show

what is your BATCH_SIZE value?

Sorry,I set max batch size to 2,It OK now ,Thank you very much!