ValueError: Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0

[GUORUIWANG]

Is Ampere GPU with cuda 11.0 a necessary condition? How to solve this error? Thank you

[aceai84]

I have encountered the same problem too.

[daiyongya]

Is Ampere GPU with cuda 11.0 a necessary condition? How to solve this error? Thank you

do you use v100? It doesn't support bf16

[bingjie3216]

Change bf16 to fp16 for non Ampere GPUs

[peter-ch]

How?

[sarankup]

My code the below

from transformers import TrainingArguments

args = TrainingArguments(
    output_dir="llama-7-int4-dolly",
    num_train_epochs=3,
    per_device_train_batch_size=6 if use_flash_attention else 4,
    gradient_accumulation_steps=2,
    gradient_checkpointing=True,
    optim="paged_adamw_32bit",
    logging_steps=10,
    save_strategy="epoch",
    learning_rate=2e-4,
    bf16=True,
    tf32=True,
    max_grad_norm=0.3,
    warmup_ratio=0.03,
    lr_scheduler_type="constant",
    disable_tqdm=True # disable tqdm since with packing values are in correct
)

I have changed them to below and it works.

    bf16=True,
    tf32=True,

[NightMachinery]

I changed bf16=True, to fp16=True in TrainingArguments. It seemed to solve that problem.

[StatsGary]

Change bf16 to fp16 for non Ampere GPUs

this answer solved it for me - no need for additional arguments, as recommended by other Giters.

[abhigarg]

I changed to bf16=false and it worked for me