TypeError: NoneType takes no arguments

[lvjin521]

I reported the following error while executing the startup script:

python main.py --config-path configs/config-dev-offload-1-4090.json --port 6006 --host 0.0.0.0

You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 Traceback (most recent call last): File "/workspace/flux-fp8-api/main.py", line 199, in main() File "/workspace/flux-fp8-api/main.py", line 159, in main app.state.model = FluxPipeline.load_pipeline_from_config_path( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/flux-fp8-api/flux_pipeline.py", line 679, in load_pipeline_from_config_path return cls.load_pipeline_from_config(config, debug=debug) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/flux-fp8-api/flux_pipeline.py", line 703, in load_pipeline_from_config flow_model = quantize_flow_transformer_and_dispatch_float8( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/workspace/flux-fp8-api/float8_quantize.py", line 490, in quantize_flow_transformer_and_dispatch_float8 swap_to_cublaslinear(flow_model) File "/usr/local/lib/python3.11/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/workspace/flux-fp8-api/float8_quantize.py", line 380, in swap_to_cublaslinear cublas_lin = CublasLinear( ^^^^^^^^^^^^^ TypeError: NoneType takes no arguments

[aredden]

Ah yeah I'll look into this, shouldn't be happening. Is caused by the https://github.com/aredden/torch-cublas-hgemm library not being installed. Should be a quick fix. I'll do it today.

[aredden]

Okay should be fixed now! From here: 153dd913d02f05023fdf3b6c24a16d737f3c1359 - let me know if there are any further issues @lvjin521

[Viper373]

Okay should be fixed now! From here: 153dd91 - let me know if there are any further issues @lvjin521

thanks! This solved my problem.

[lvjin521]

Wow, thank you so much for your optimization, I have no problem now, you are great