Sampling on 3090 not using full GPU

[DHOFM]

Hi, could anybody tell me how to parametrize the sampling script for oner or two RTX 3090. I just installed everything it gave it a try using the example: python jukebox/sample.py --model=5b_lyrics --name=sample_5b --levels=3 --sample_length_in_seconds=20 \ --total_sample_length_in_seconds=180 --sr=44100 --n_samples=6 --hop_fraction=0.5,0.5,0.125

I also changed the max_batch_size and the chunk_size in the sampling script with less effect. The Machine has 192 GB RAM and two RTX 3090 GPUs. This is what I get with the default parameters:

165W / 350W | 9180MiB / 24268MiB | 34%

+-------------------------------+----------------------+

153W / 350W | 10532MiB / 24268MiB | 28%

Any suggestions? Thanks.

Kind regards, Dirk

[DHOFM]

Update: Increasing the max_batch_size and the n_samples will use the whole GPU Memory but it will never have 100% Util except some small peaks. But The GPUs will use more than 300 Watts so I would guess it's because of inferencing, that's only using 60%

[surpscare]

Update: Increasing the max_batch_size and the n_samples will use the whole GPU Memory but it will never have 100% Util except some small peaks. But The GPUs will use more than 300 Watts so I would guess it's because of inferencing, that's only using 60%

Good day, can you please share your steps to set up an environment and jupyter notebook to run it locally, and also which Linux distro you are using? Because I ran an issue while performing the default installation guide and running an Interacting-with_Jukebox.ipynb file - at the step to generate level_2 it spits out RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasGemmEx ( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16F, lda, b, CUDA_R_16F, ldb, &fbeta, c, CUDA_R_16F, ldc, CUDA_R_32F, CUBLAS_GEMM_DFALT_TENSOR_OP)