Can Ollama be Packaged with Tranformerlab's AppImag in Linux?

[mbennette68]

Hello,

Please excuse the post if this is not the right place to post this question.

Is it possible to package Ollama with the Transformerlab Appimage rather than installing it as a separate service on Linux. This will greatly reduce resource usage on Laptops, even if the hardware comes with a NVidia discrete card.

If packaging Ollama is not part of the roadmap, is there any way to install it somewhere inside the ~/.transformerlab directory and have it activated only when the Inference Server is set to 'Ollama Server' or when the model is run implying that when the model stops running, Ollama will also be stopped, again freeing resources?

Thank you, MB

[deep1401]

Hello, We do not currently plan to distribute Ollama with our software and since ollama is a separate app in itself, I'm afraid the things you suggested won't be possible directly. However, I can suggest that you can maybe try using the llama cpp server instead? We have recently upgraded it in our latest release to work on the newer versions and it should now be at-par with other plugins. I would strongly suggest using that if you face resource constraints. Please let me know if I missed something while answering your query!

[mbennette68]

Hello,

I have installed the "llama cpp" server and am trying to download the new "Gemma 3 QAT 1B IT (q4_0 GGUF)" model. I noticed that the download indicator, in this instance, does not seem to show progress but I assume that the download is proceeding in the background. I will see how it goes.

I appreciate the feedback, MB

[dadmobile]

Did this finish? Since it's a small model it should have downloaded quickly so I wonder if something is wrong. If it didn't work perhaps you can share the recent contents of ~/.transformerlab/transformerlab.log?

[mbennette68]

Did this finish? Since it's a small model it should have downloaded quickly so I wonder if something is wrong. If it didn't work perhaps you can share the recent contents of ~/.transformerlab/transformerlab.log?

Actually no. I installed the llama cpp and then tried to download the model but it did not download it. I cancelled, closed the app and restarted it again, but then it tried to update itself twice. After that, it closed down, and did not restart after the update. I had to do a manual restart. I tried again to download the model but after ten minutes it did not download the model so I canceled.

And now as I typed this message, a popup came up to the front of the screen letting me know that it failed to download.

Attached is the transfermerlab.log file as it is at this moment. Towards the end there seem to be some errors but not sure if that's supposed to happen with llama cpp.

transformerlab_04-29-25.log

FYI, "TinyLlama/TinyLlama-1.1B-Chat-v1.0" works without issues, but I still don't recall seeing a download progress indicator when I downloaded it few days ago.

Thank you, MB

[dadmobile]

For the GGUF model, were you downloading one from our gallery or trying to download by entering the model ID?

It looks like your llama_cpp build failed. The error looks like it couldn't find a wheel and tried to compile but you don't have a C compiler on your system. I will open in a separate issue. Part of why we recommend folks use Ollama (even though it requires installing it separately, as you mentioned) is because of these kind of issues with installing llama.cpp on its own.

[mbennette68]

For the GGUF model, were you downloading one from our gallery or trying to download by entering the model ID? Yes, I was downloading from your gallery, directly not from a local file.

I understand the reasoning about using Ollama as a stand-alone package.

Appreciate the feedback, MB

[dadmobile]

OK I tried with this model...it sort of froze but then the download box disappeared and the model showed up in my list as gemma-3-1b-it-q4_0.gguf. Possibly because the file is so small it is downloading quicker than the UI is updating.

Can you look in Local Models and see if there's a model with that name (I realize it's different than the download name...looks like a bug!). If not, you could try downloading but first open up the console at the bottom of the window to see what it says?

If you really want to know what's going on...you could look at the ~/.transformerlab/workspace/models directory. It should be creating a subdirectly called gemma-3-1b-it-q4_0.gguf and the files associated with the download should be in there.

[mbennette68]

If you really want to know what's going on...you could look at the ~/.transformerlab/workspace/models directory. It should be creating a subdirectly called gemma-3-1b-it-q4_0.gguf and the files associated with the download should be in there.

Ok, I tried the above before doing anything else to see what was in there and this is it:

Can you look in Local Models and see if there's a model with that name (I realize it's different than the download name...looks like a bug!). If not, you could try downloading but first open up the console at the bottom of the window to see what it says?

Ok so this is what I have in Local Models per your request and doing anything else:

OK I tried with this model...it sort of froze but then the download box disappeared and the model showed up in my list as gemma-3-1b-it-q4_0.gguf. Possibly because the file is so small it is downloading quicker than the UI is updating.

So, I tried look for "gemma-3-1b-it-q4_0.gguf but I could not find anything by searching using the model name as spelled above and shown below:

Though there exist a great good number of models named Gemma, none matches the specified name, at least not when looking from my end :(

Please let me know if I did not follow the steps correctly to reproduce the results you asked for.

Thank you, MB

[deep1401]

Hi, Sorry for missing on replying to this. Could you also send the file called "local_server.log" and "transformerlab.log" located at ~/.transformerlab so I can look in and verify. Also a suggestion to setup your huggingface token in the Settings tab which might also be causing these weird errors

[mbennette68]

Hello,

No worries.

Could you also send the file called "local_server.log" and "transformerlab.log" located at ~/.transformerlab so I can look in and verify.

local_server_04-29-25.log transformerlab_04-29-25.log

Attached you will find the two requested files.

Also a suggestion to setup your huggingface token in the Settings tab which might also be causing these weird errors

Basically, you are saying that for me to download GGUF Models, I have to add my Huggin Face credentials, which makes sense since the models are stored in gated repositories on Hugging Face, requiring authentication. Also, it looks like CMake is looking for GCC at a different location based on the CXX environment variable's content even though GCC can be found from the terminal:

which gcc /usr/bin/gcc

which g++ /usr/bin/g++

I've yet to setup any huggingface token. I'm a little hesitant to do so as am not sure how stable my system will run. The pc where transformerlab is hosted is my daily driver pc, at the moment, so don't want to cause any unneeded instabilities. With that being said, I guess I have no choice if I want to download the GGUF models. There are so many settings when generating an access token, it's a bit overwhelming.

Update:

• I added a Read Access Token.
• I was able to download "Llama-3.2-1B-Instruct-Q6_K.gguf" without issues just not able to run it.
• "Gemma 3 QAT 1B IT (q4_0 GGUF).gguf" just can't be downloaded from Model Zoo. Don't know why?

Thank you, MB

[dadmobile]

Sorry for the slow reply...we should file this issue as a separate bug so it doesn't get lost.

TLDR: This model is gated but we didn't mark it as gated. To get it to work you have to add a huggingface token to our settings page.

Details:

1. The first problem is that there is an error downloading the model because it is gated. But our app does not have it marked as gated for some reason, and therefore doesn't alert you. In the log I can see:

Cannot access gated repo for url https://huggingface.co/google/gemma-3-1b-it-qat-q4_0-gguf/resolve/main/gemma-3-1b-it-q4_0.gguf. Access to model google/gemma-3-1b-it-qat-q4_0-gguf is restricted.

2. The second problem is that instead of telling you that the model is gated it appears to try to download the model anyways, which fails. I see this over and over in your log:

Model Download Progress: 0.00%

I assume in the UI this just looks like nothing is happening.

I will file this as a separate bug and link you.

[dadmobile]

Actually no need for separate issue. I've fixed the model to be correctly labelled for now. The second problem has to do with how we add models to our gallery and is being addressed by other work we are doing. Closing for now.