Google Colab Jupyter Notebook Broken

[c4ch3c4d3]

Unfortunately, when trying to run Transfomerlab via Google Colab notebook, Jupyter side runs correctly, and plugins install properly, models fail to load. It might have something to do with the virtual environments that Transformerlab automatically creates via uv. I've spent some time trying to troubleshoot, but have had no success solving the problem myself. The error for both llama-cpp and Fastchat servers is always related to torch missing, as shown below:

🏃 Loading Inference Server for Llama-3.2-1B-Instruct-Q6_K.gguf with {'inferenceEngine': 'llama_cpp_server', 'inferenceEngineFriendlyName': 'Llama-cpp Server'}
Traceback (most recent call last):
  File "/root/.transformerlab/src/transformerlab/plugin_sdk/plugin_harness.py", line 26, in <module>
    import main
  File "/root/.transformerlab/workspace/plugins/llama_cpp_server/main.py", line 15, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'
Error executing plugin: No module named 'torch'
ModuleNotFoundError means a Python package is missing. This is usually fixed by reinstalling the plugin
Error loading model: Llama-3.2-1B-Instruct-Q6_K.gguf with exit code 1
warning: The `--system` flag has no effect, a system Python interpreter is always used in `uv venv`
Using CPython 3.11.13 interpreter at: /usr/bin/python3

This is despite torch showing properly within the Computer > Python Libraries view (2.7.0+cu128). Thank you!

[deep1401]

Hi, Thanks for reporting this! This was also recently reported on our Discord server and we're working on making this work. I'll update the status here once we have a fix

[dadmobile]

Update I posted in Discord:

We've spent a bunch of time investigating this and we're kind of stuck as Google Colab doesn't seem to support the way we create virtual environments for each of our plugins (because the colab is itself running in a virtual environment). We are exploring options, but thinking the best direction might be to create more of these "easy start" guides for other more powerful platforms (Colab is great for sharing but super underpowered for most of things Transformer Lab can do).

For now we are going to take down our Colab doc and try to replace with something for salad.com or runpod or some other more powerful alternative. (BTW if you haven't tried salad.com I recommend checking out)