axolotl Installation is difficult in different systems. Can you make it run in colab?

What piece of documentation is affected?

I tried installing this package in Windows and I miserably failed. I am trying to make it run in google colab, and I still have big problems. The suggested installation steps did not work for me. Could you have in the examples one notebook that would install the package and run one example using colab? That would be great. Thanks!

What part(s) of the article would you like to see updated?

Either the installation steps or having an exemplary notebook that runs in colab

Additional Information

No response

Acknowledgements

[X] My issue title is concise, descriptive, and in title casing.
[X] I have searched the existing issues to make sure this feature has not been requested yet.
[X] I have provided enough information for the maintainers to understand and evaluate this request.

Jan 23 '24 22:01 Vostredamus

Hi! I have it working in a colab notebook, can share it later! Would you like to have it as an example in the repo as well @NanoCode012 ? otherwise I will just drop it here :)

Jan 24 '24 15:01 JohanWork

Great! Both options are good, leaving a copy in the examples folder is better for other users...

Thank you!

El mié., 24 ene. 2024 16:31, JohanWork @.***> escribió:

Hi! I have it working in a notebook, can share it later! Would you like to have it as an example in the repo as well @NanoCode012 https://github.com/NanoCode012 ? otherwise I will just drop it here :)

— Reply to this email directly, view it on GitHub https://github.com/OpenAccess-AI-Collective/axolotl/issues/1180#issuecomment-1908368562, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFR6UZZHCGNWJEUV4Q2YFLDYQESNVAVCNFSM6AAAAABCHYUQ76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYGM3DQNJWGI . You are receiving this because you authored the thread.Message ID: @.***>

Jan 24 '24 16:01 Vostredamus

Let's see what Nano says, but for know here you have the code in plain python. It is a colab notebook downloaded as python. Let me know how it works for you :) @Vostredamus

# -*- coding: utf-8 -*-
"""axolotl.ipynb

Automatically generated by Colaboratory.

Original file is located at
    https://colab.research.google.com/drive/1Ta60zvd8_omBmhJedMeLNn2_UlMYXJRa

## Make sure you have a GPU(You will need it ;)), a T4(free tier) is enough.
"""

import torch
assert (torch.cuda.is_available()==True)

"""## Install Axolotl, deepspeed and flash attention"""

!pip install -e git+https://github.com/OpenAccess-AI-Collective/axolotl#egg=axolotl
!pip install flash-attn=="2.5.0"
!pip install deepspeed=="0.13.1"

"""## Create a config file for your training"""

import yaml

# Your YAML string
yaml_string = """
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
is_llama_derived_model: true

load_in_8bit: false
load_in_4bit: true
strict: false

datasets:
  - path: mhenrichsen/alpaca_2k_test
    type: alpaca
dataset_prepared_path:
val_set_size: 0.05
output_dir: ./qlora-out

adapter: qlora
lora_model_dir:

sequence_len: 1096
sample_packing: true
pad_to_sequence_len: true

lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:

wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

mlflow_experiment_name: test-test

gradient_accumulation_steps: 1
micro_batch_size: 1
num_epochs: 4
optimizer: paged_adamw_32bit
lr_scheduler: cosine
learning_rate: 0.0002

train_on_inputs: false
group_by_length: false
bf16: false
fp16: true
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: false

warmup_steps: 10
evals_per_epoch: 4
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
"""

# Convert the YAML string to a Python dictionary
yaml_dict = yaml.safe_load(yaml_string)

# Specify your file path
file_path = 'test_axolotl.yaml'

# Write the YAML file
with open(file_path, 'w') as file:
    yaml.dump(yaml_dict, file)

"""##Start the training with Axolotl"""

!accelerate launch -m axolotl.cli.train /content/test_axolotl.yaml

Jan 24 '24 16:01 JohanWork

@JohanWork , cool! That would be appreciated.

One thing I want to point out is that: perhaps, we can make it point to one of the configs within example folder as default and have a separate section for custom yamls.

Caseus also made one way back, but I guess it didn't get merged: https://colab.research.google.com/drive/1ts9Ar63sFK49oSz3dcw2EkivL0ZJesKi#scrollTo=iAZH0dQRFLNm

For my own debugging, I have one quite similar to yours (minus the FA, I seemed to have issues with that).

Jan 24 '24 23:01 NanoCode012

I saw that the pr actually all ready is merged. I am actually more in favour for having the yaml in the notebook. The reasons for that is that I makes it super easy for the user, and very clear how it works which I overall think is the benefit of colab. When using a path reference it will make it a little more complicated for total beginners. But happy to make an update in a new pr if you want!

Jan 25 '24 06:01 JohanWork

@JohanWork , ah it's alright. This is a good start.

@Vostredamus , could you try the colab that Johan has added?

Jan 25 '24 07:01 NanoCode012

It works very well! I would also leave the config file in the code, as it makes it very clear and self-sufficient. What it did not work for me yet is activating the deepspeed, but that is another issue...

El jue., 25 ene. 2024 8:07, NanoCode012 @.***> escribió:

@JohanWork https://github.com/JohanWork , ah it's alright. This is a good start.

@Vostredamus https://github.com/Vostredamus , could you try the colab that Johan has added?

— Reply to this email directly, view it on GitHub https://github.com/OpenAccess-AI-Collective/axolotl/issues/1180#issuecomment-1909471457, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFR6UZ7ZWCG3BOVYUMER7RDYQIABZAVCNFSM6AAAAABCHYUQ76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBZGQ3TCNBVG4 . You are receiving this because you were mentioned.Message ID: @.***>

Jan 25 '24 14:01 Vostredamus

I have create a new notebook here, that is colab+deepspeed. Try it out @Vostredamus and let me know if it works for you as well. Should I add it to the official repo as well @NanoCode012 ?

Jan 27 '24 07:01 JohanWork

@JohanWork

Hello your colab worked well until a couple of days ago, now it requires another version of xformers?!

this seems to work in colab (downgrading xformers to 0.0.23) %cd {base_dir} !git clone https://github.com/OpenAccess-AI-Collective/axolotl %cd {base_dir}axolotl

!pip3 install packaging !pip3 install xformers=='0.0.23' !pip3 install -e '.[flash-attn,deepspeed]'

using your instructions, I get these errors:



Obtaining axolotl from git+https://github.com/OpenAccess-AI-Collective/axolotl#egg=axolotl
  Updating ./src/axolotl clone
  Running command git fetch -q --tags
  Running command git reset --hard -q d113331e9a590c09faff3d00e33be06aec367515
  Preparing metadata (setup.py) ... done
Collecting peft@ git+https://github.com/huggingface/peft.git (from axolotl)
  Cloning https://github.com/huggingface/peft.git to /tmp/pip-install-v7759mvw/peft_019aec82764244b69c7265ad576aa9a5
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft.git /tmp/pip-install-v7759mvw/peft_019aec82764244b69c7265ad576aa9a5
  Resolved https://github.com/huggingface/peft.git to commit fff24008ebf6ee24fe869af45fccb77ed58294aa
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging==23.2 in /usr/local/lib/python3.10/dist-packages (from axolotl) (23.2)
Collecting transformers==4.37.0 (from axolotl)
  Using cached transformers-4.37.0-py3-none-any.whl (8.4 MB)
Collecting tokenizers==0.15.0 (from axolotl)
  Using cached tokenizers-0.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
Collecting bitsandbytes>=0.41.1 (from axolotl)
  Using cached bitsandbytes-0.42.0-py3-none-any.whl (105.0 MB)
Collecting accelerate==0.26.1 (from axolotl)
  Using cached accelerate-0.26.1-py3-none-any.whl (270 kB)
Collecting addict (from axolotl)
  Using cached addict-2.4.0-py3-none-any.whl (3.8 kB)
Collecting fire (from axolotl)
  Using cached fire-0.5.0.tar.gz (88 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: PyYAML>=6.0 in /usr/local/lib/python3.10/dist-packages (from axolotl) (6.0.1)
Collecting datasets>=2.15.0 (from axolotl)
  Using cached datasets-2.16.1-py3-none-any.whl (507 kB)
Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.1.99)
Collecting wandb (from axolotl)
  Using cached wandb-0.16.2-py3-none-any.whl (2.2 MB)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.7.0)
Collecting optimum==1.16.2 (from axolotl)
  Using cached optimum-1.16.2-py3-none-any.whl (402 kB)
Collecting hf_transfer (from axolotl)
  Using cached hf_transfer-0.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5 MB)
Collecting colorama (from axolotl)
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from axolotl) (0.58.1)
Collecting numpy>=1.24.4 (from axolotl)
  Using cached numpy-1.26.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
Collecting mlflow (from axolotl)
  Using cached mlflow-2.10.0-py3-none-any.whl (19.5 MB)
Collecting evaluate==0.4.0 (from axolotl)
  Using cached evaluate-0.4.0-py3-none-any.whl (81 kB)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from axolotl) (1.11.4)
Requirement already satisfied: scikit-learn==1.2.2 in /usr/local/lib/python3.10/dist-packages (from axolotl) (1.2.2)
Requirement already satisfied: pynvml in /usr/local/lib/python3.10/dist-packages (from axolotl) (11.5.0)
Collecting art (from axolotl)
  Using cached art-6.1-py3-none-any.whl (599 kB)
Collecting fschat==0.2.34 (from axolotl)
  Using cached fschat-0.2.34-py3-none-any.whl (220 kB)
Collecting gradio==3.50.2 (from axolotl)
  Using cached gradio-3.50.2-py3-none-any.whl (20.3 MB)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from axolotl) (2.15.1)
Collecting s3fs (from axolotl)
  Using cached s3fs-2023.12.2-py3-none-any.whl (28 kB)
Requirement already satisfied: gcsfs in /usr/local/lib/python3.10/dist-packages (from axolotl) (2023.6.0)
Collecting trl>=0.7.9 (from axolotl)
  Using cached trl-0.7.10-py3-none-any.whl (150 kB)
Requirement already satisfied: torch==2.1.0+cu121 in /usr/local/lib/python3.10/dist-packages (from axolotl) (2.1.0+cu121)
Collecting xformers>=0.0.23 (from axolotl)
  Using cached xformers-0.0.24-cp310-cp310-manylinux2014_x86_64.whl (218.2 MB)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.26.1->axolotl) (5.9.5)
Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate==0.26.1->axolotl) (0.20.3)
Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.26.1->axolotl) (0.4.2)
Collecting dill (from evaluate==0.4.0->axolotl)
  Using cached dill-0.3.8-py3-none-any.whl (116 kB)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl) (1.5.3)
Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl) (2.31.0)
Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl) (4.66.1)
Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl) (3.4.1)
Collecting multiprocess (from evaluate==0.4.0->axolotl)
  Using cached multiprocess-0.70.16-py310-none-any.whl (134 kB)
Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl) (2023.6.0)
Collecting responses<0.19 (from evaluate==0.4.0->axolotl)
  Using cached responses-0.18.0-py3-none-any.whl (38 kB)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.34->axolotl) (3.9.1)
Collecting fastapi (from fschat==0.2.34->axolotl)
  Using cached fastapi-0.109.0-py3-none-any.whl (92 kB)
Collecting httpx (from fschat==0.2.34->axolotl)
  Using cached httpx-0.26.0-py3-none-any.whl (75 kB)
Collecting markdown2[all] (from fschat==0.2.34->axolotl)
  Using cached markdown2-2.4.12-py2.py3-none-any.whl (41 kB)
Collecting nh3 (from fschat==0.2.34->axolotl)
  Using cached nh3-0.2.15-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
Requirement already satisfied: prompt-toolkit>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.34->axolotl) (3.0.43)
Requirement already satisfied: pydantic<2,>=1 in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.34->axolotl) (1.10.14)
Requirement already satisfied: rich>=10.0.0 in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.34->axolotl) (13.7.0)
Collecting shortuuid (from fschat==0.2.34->axolotl)
  Using cached shortuuid-1.0.11-py3-none-any.whl (10 kB)
Collecting tiktoken (from fschat==0.2.34->axolotl)
  Using cached tiktoken-0.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
Collecting uvicorn (from fschat==0.2.34->axolotl)
  Using cached uvicorn-0.27.0.post1-py3-none-any.whl (60 kB)
Collecting aiofiles<24.0,>=22.0 (from gradio==3.50.2->axolotl)
  Using cached aiofiles-23.2.1-py3-none-any.whl (15 kB)
Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (4.2.2)
Collecting ffmpy (from gradio==3.50.2->axolotl)
  Using cached ffmpy-0.3.1.tar.gz (5.5 kB)
  Preparing metadata (setup.py) ... done
Collecting gradio-client==0.6.1 (from gradio==3.50.2->axolotl)
  Using cached gradio_client-0.6.1-py3-none-any.whl (299 kB)
Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (6.1.1)
Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (3.1.3)
Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (2.1.4)
Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (3.7.1)
Collecting orjson~=3.0 (from gradio==3.50.2->axolotl)
  Using cached orjson-3.9.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (9.4.0)
Collecting pydub (from gradio==3.50.2->axolotl)
  Using cached pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting python-multipart (from gradio==3.50.2->axolotl)
  Using cached python_multipart-0.0.6-py3-none-any.whl (45 kB)
Collecting semantic-version~=2.0 (from gradio==3.50.2->axolotl)
  Using cached semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.50.2->axolotl) (4.5.0)
Collecting websockets<12.0,>=10.0 (from gradio==3.50.2->axolotl)
  Using cached websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
Collecting coloredlogs (from optimum==1.16.2->axolotl)
  Using cached coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from optimum==1.16.2->axolotl) (1.12)
Requirement already satisfied: transformers[sentencepiece]>=4.26.0 in /usr/local/lib/python3.10/dist-packages (from optimum==1.16.2->axolotl) (4.35.2)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn==1.2.2->axolotl) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn==1.2.2->axolotl) (3.2.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch==2.1.0+cu121->axolotl) (3.13.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.1.0+cu121->axolotl) (3.2.1)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch==2.1.0+cu121->axolotl) (2.1.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.37.0->axolotl) (2023.6.3)
Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.15.0->axolotl) (10.0.1)
Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets>=2.15.0->axolotl) (0.6)
Collecting dill (from evaluate==0.4.0->axolotl)
  Using cached dill-0.3.7-py3-none-any.whl (115 kB)
Collecting tyro>=0.5.11 (from trl>=0.7.9->axolotl)
  Using cached tyro-0.7.0-py3-none-any.whl (79 kB)
INFO: pip is looking at multiple versions of xformers to determine which version is compatible with other requirements. This could take a while.
Collecting xformers>=0.0.23 (from axolotl)
  Using cached xformers-0.0.23.post1-cp310-cp310-manylinux2014_x86_64.whl (213.0 MB)
  Using cached xformers-0.0.23-cp310-cp310-manylinux2014_x86_64.whl (213.0 MB)
Collecting trl>=0.7.9 (from axolotl)
  Using cached trl-0.7.9-py3-none-any.whl (141 kB)
ERROR: Cannot install axolotl and axolotl==0.4.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    axolotl 0.4.0 depends on torch==2.1.0+cu121
    accelerate 0.26.1 depends on torch>=1.10.0
    optimum 1.16.2 depends on torch>=1.11
    trl 0.7.9 depends on torch>=1.4.0
    xformers 0.0.24 depends on torch==2.2.0
    axolotl 0.4.0 depends on torch==2.1.0+cu121
    accelerate 0.26.1 depends on torch>=1.10.0
    optimum 1.16.2 depends on torch>=1.11
    trl 0.7.9 depends on torch>=1.4.0
    xformers 0.0.23.post1 depends on torch==2.1.2
    axolotl 0.4.0 depends on torch==2.1.0+cu121
    accelerate 0.26.1 depends on torch>=1.10.0
    optimum 1.16.2 depends on torch>=1.11
    trl 0.7.9 depends on torch>=1.4.0
    xformers 0.0.23 depends on torch==2.1.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
Requirement already satisfied: flash-attn==2.5.0 in /usr/local/lib/python3.10/dist-packages (2.5.0)
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (2.1.0+cu121)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (0.7.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (23.2)
Requirement already satisfied: ninja in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.5.0) (1.11.1.1)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.13.1)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (4.5.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.2.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (3.1.3)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (2023.6.0)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.5.0) (2.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->flash-attn==2.5.0) (2.1.4)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->flash-attn==2.5.0) (1.3.0)
Requirement already satisfied: deepspeed==0.13.1 in /usr/local/lib/python3.10/dist-packages (0.13.1)
Requirement already satisfied: hjson in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (3.1.0)
Requirement already satisfied: ninja in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (1.11.1.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (1.23.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (23.2)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (5.9.5)
Requirement already satisfied: py-cpuinfo in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (9.0.0)
Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (1.10.14)
Requirement already satisfied: pynvml in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (11.5.0)
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (2.1.0+cu121)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from deepspeed==0.13.1) (4.66.1)
Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from pydantic->deepspeed==0.13.1) (4.5.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->deepspeed==0.13.1) (3.13.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->deepspeed==0.13.1) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->deepspeed==0.13.1) (3.2.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->deepspeed==0.13.1) (3.1.3)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->deepspeed==0.13.1) (2023.6.0)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch->deepspeed==0.13.1) (2.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->deepspeed==0.13.1) (2.1.4)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->deepspeed==0.13.1) (1.3.0)

Jan 31 '24 19:01 g-i-o-r-g-i-o

I don’t think it needs a downgrade, I think pytorch needs to be updated, or at least I think it is a better fix. @g-i-o-r-g-i-o

xformers 0.0.23 depends on torch==2.1.1

Will updated!

Feb 01 '24 20:02 JohanWork

I would recommend going to torch 2.1.2

Feb 01 '24 22:02 winglian

Hey, sorry for taking this long to test it. I confirm that I find the same errors, and the training will not run due to them. A pity....

Feb 03 '24 22:02 Vostredamus

The example is updated and should work again since this is merged :) https://github.com/OpenAccess-AI-Collective/axolotl/pull/1247#event-11717629588 So try again @Vostredamus it should work now!

Feb 06 '24 19:02 JohanWork

Is still not working (see attachment)

On Tue, Feb 6, 2024 at 8:39 PM JohanWork @.***> wrote:

The example is updated and should work again since this is merged :) #1247 (comment) https://github.com/OpenAccess-AI-Collective/axolotl/pull/1247#event-11717629588 So try again @Vostredamus https://github.com/Vostredamus it should work now!

— Reply to this email directly, view it on GitHub https://github.com/OpenAccess-AI-Collective/axolotl/issues/1180#issuecomment-1930628831, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFR6UZ6T5TEPTEPO645E5Y3YSKBFXAVCNFSM6AAAAABCHYUQ76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZQGYZDQOBTGE . You are receiving this because you were mentioned.Message ID: @.***>

Feb 15 '24 16:02 Vostredamus

I just cloned it from https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/examples/colab-notebooks/colab-axolotl-example.ipynb and tried it in colab. For me it works fine, are you using the latest update colab notebook? @Vostredamus

Feb 15 '24 20:02 JohanWork

I missed the pip install torch=="2.1.2". Now it works Many thanks!

On Thu, Feb 15, 2024 at 9:34 PM JohanWork @.***> wrote:

I just cloned it from https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/examples/colab-notebooks/colab-axolotl-example.ipynb and tried it in colab. For me it works fine, are you using the latest update colab notebook? @Vostredamus https://github.com/Vostredamus

— Reply to this email directly, view it on GitHub https://github.com/OpenAccess-AI-Collective/axolotl/issues/1180#issuecomment-1947298032, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFR6UZZG3H3XR2KYZAP2U3TYTZWNLAVCNFSM6AAAAABCHYUQ76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGI4TQMBTGI . You are receiving this because you were mentioned.Message ID: @.***>

Feb 15 '24 21:02 Vostredamus

axolotl axolotl copied to clipboard

Installation is difficult in different systems. Can you make it run in colab?

What piece of documentation is affected?

What part(s) of the article would you like to see updated?

Additional Information

Acknowledgements

axolotl
axolotl copied to clipboard