Calvin Pelletier

"Bus error (core dumped)" when saving recipe state after restarting training

1

(machine: Fedora Linux, 1xH100 gpu) Run `tune run lora_finetune_single_device --config llama3/8B_lora_single_device seed=0 epochs=4 max_steps_per_epoch=2` Then interrupt after one epoch has been successfully saved to disk Then run `tune run lora_finetune_single_device...

DoRA

3

#### Context Adding support for DoRA: https://arxiv.org/abs/2402.09353 Also refactoring `LoRALinear`. The adapter logic is now encapsulated in a submodule `LowRankAdapter` while the base layer logic remains in `LoRALinear`. This lets...

[RFC] Image Generation Dataset

3

# Overview This is an RFC regarding how we should support datasets for finetuning text-conditioned image generation models. A basic data pipeline for this would be: 1. Load the JSON/CSV/TSV/Parquet/LMDB/etc....

[WIP] Flux finetuning

2

TODO