who to run aqlm model

[werruww]

who to run aqlm model??????

[werruww]

ISTA-DASLab/Llama-3.2-1B-AQLM-PV-2Bit-2x8

[werruww]

Some models require installing special programs pip install aqlm[gpu,cpu] How do I install it in the program?

[werruww]

???????

pip install optimum

[werruww]

How to install programs in the program؟

[werruww]

Is it possible to install specific programs in the program?

[werruww]

How do I get to miniconda3transformerlab-app

[dadmobile]

Depending on what you are trying to do, the main ways you would extend the program would be:

• add a plugin to extend functionality. Plugins have a setup script that installs required libraries and then can add functionality to various parts of the application (serving models, training, converting between formats...see plugins tab)
• editing existing plugins to add libraries and then update the code to use those libraries

If you just want to install libraries to the environment you can do so but you need to be in the right conda environment. If you run conda info --envs you will see your conda list...there should be an environment with a name like Users/username/.transformerlab/envs/transformerlab.

If you want to change core functionality in the engine that runs the app, you might want to actually clone the API. In that case I suggest doing "Manual Step-by-step install" on this page: https://transformerlab.ai/docs/install/advanced-install

[werruww]

I searched and did not find. Is there a way to add specific libraries?

[dadmobile]

As I mentioned, if you only want to install a library you would use conda on the command line like conda activate <environment>. You get by doing conda info --envs.

But usually, if you are installing a library you also want to add functionality to use it. To do that you would create or edit a plugin.

[werruww]

(base) C:\Windows\system32>conda info --envs

conda environments:

base * C:\ProgramData\anaconda3

(base) C:\Windows\system32>

[werruww]

[werruww]

win10

[dadmobile]

If you are running on Windows, the engine is running in WSL. So in order to interact with the environment you will have to work inside of WSL.

[werruww]

I am in Windows without wsl

[werruww]

I have run unsloth/Phi-3-mini-4k-instruct-v0-bnb-4bit Does it support other compressed models?

[werruww]

[werruww]

I use Anaconda and the program uses Mini Conda3 and I don't know where its path is

[werruww]

How to install libraries manually

[werruww]

in win 10 not wsl

[deep1401]

I tried to do this by installing aqlm, but it looks like there are some build issues of aqlm with cuda 12.8 and torch 2.8:

Error occurred in generation {'text': '**NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.**\n\n(Error building extension \'codebook_cuda\': [1/3] /home/transformerlab/.transformerlab/envs/transformerlab/bin/x86_64-conda-linux-gnu-c++ -MMD -MF cuda_kernel.o.d -DTORCH_EXTENSION_NAME=codebook_cuda -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\\"_gcc\\" -DPYBIND11_STDLIB=\\"_libstdcpp\\" -DPYBIND11_BUILD_ABI=\\"_cxxabi1018\\" -isystem /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include -isystem /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/transformerlab/.transformerlab/envs/transformerlab/include -isystem /home/transformerlab/.transformerlab/envs/transformerlab/include/python3.11 -fPIC -std=c++17 -c /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/aqlm/inference_kernels/cuda_kernel.cpp -o cuda_kernel.o \nFAILED: cuda_kernel.o \n/home/transformerlab/.transformerlab/envs/transformerlab/bin/x86_64-conda-linux-gnu-c++ -MMD -MF cuda_kernel.o.d -DTORCH_EXTENSION_NAME=codebook_cuda -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\\"_gcc\\" -DPYBIND11_STDLIB=\\"_libstdcpp\\" -DPYBIND11_BUILD_ABI=\\"_cxxabi1018\\" -isystem /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include -isystem /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/transformerlab/.transformerlab/envs/transformerlab/include -isystem /home/transformerlab/.transformerlab/envs/transformerlab/include/python3.11 -fPIC -std=c++17 -c /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/aqlm/inference_kernels/cuda_kernel.cpp -o cuda_kernel.o \nIn file included from /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/c10/cuda/CUDAGraphsC10Utils.h:3,\n                 from /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/c10/cuda/CUDACachingAllocator.h:4,\n                 from /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/c10/cuda/impl/CUDAGuardImpl.h:8,\n                 from /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/c10/cuda/CUDAGuard.h:7,\n                 from /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/aqlm/inference_kernels/cuda_kernel.cpp:3:\n/home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/c10/cuda/CUDAStream.h:3:10: fatal error: cuda_runtime_api.h: No such file or directory\n    3 | #include <cuda_runtime_api.h>\n      |          ^~~~~~~~~~~~~~~~~~~~\ncompilation terminated.\n[2/3] /home/transformerlab/.transformerlab/envs/transformerlab/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_kernel.cuda.o.d -ccbin /home/transformerlab/.transformerlab/envs/transformerlab/bin/x86_64-conda-linux-gnu-cc -DTORCH_EXTENSION_NAME=codebook_cuda -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\\"_gcc\\" -DPYBIND11_STDLIB=\\"_libstdcpp\\" -DPYBIND11_BUILD_ABI=\\"_cxxabi1018\\" -isystem /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include -isystem /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/transformerlab/.transformerlab/envs/transformerlab/include -isystem /home/transformerlab/.transformerlab/envs/transformerlab/include/python3.11 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options \'-fPIC\' -std=c++17 -c /home/transformerlab/.transformerlab/workspace/plugins/fastchat_server/venv/lib/python3.11/site-packages/aqlm/inference_kernels/cuda_kernel.cu -o cuda_kernel.cuda.o \nnvcc warning : incompatible redefinition for option \'compiler-bindir\', the last value of this option was used\nninja: build stopped: subcommand failed.\n)', 'error_code': 50001}