alg_con

flux-demo failure of TensorRT 10.5 when running a single L40 GPU, how to implement 2-GPUs with L40

3

Currently, when I run Flux on a device with a single L40 GPU, I encounter an OutOfMemory error. I found another device L40 with two GPUs. How can I implement...

Does Flux not support int8?

1

When I use the command " python demo_txt2img_flux.py prompt \ --version "flux.1-dev" \ --int8 \ " run the code, the engine file size is the same with float16, how to...

flux model engine_from_bytes(bytes_from_path(self.engine_path)) OutOfMemory

2

from polygraphy.backend.trt import engine_from_bytes; when run engine_from_bytes(bytes_from_path(self.engine_path)) OutOfMemory on L40 with 1gpu with flux-dev,how to solve

### 🚀 The feature, motivation and pitch Currently, does this project support Flux? If there are only two GPUs, each with 24GB of memory, is it possible to perform inference?...