OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root

[immodalmul]

Geting this error while using on modal.com

[prp-e]

I get the same.

[EmmanuelMr18]

This happens to me when having quantize: true, if you disable it works perfectly. The problem is that looks to be using ~40GB of VRAM, so you will need one of the high end GPUs, which means more money.

You would rather not use those high end GPUs?

To fix this issue, you can use a different image. By default the run_modal.py comes with modal.Image.debian_slim(python_version="3.11")

But you can modify it to modal.Image.from_registry("nvidia/cuda:12.4.0-devel-ubuntu22.04", add_python="3.11")

Will look something like this:

modal.Image.from_registry("nvidia/cuda:12.4.0-devel-ubuntu22.04", add_python="3.11")
    # install required system and pip packages, more about this modal approach: https://modal.com/docs/examples/dreambooth_app
    .apt_install("libgl1", "libglib2.0-0")
    .pip_install("cupy-cuda12x")
    .pip_install(
        "python-dotenv",
        ...
    }

I'm proposing this change here: https://github.com/ostris/ai-toolkit/pull/246