Zak-Sing

I notice some functions of SVGCommandArc are not implemented yet, is there any plan to update the code and finish those functions? I put the related line below ~ https://github.com/alexandre01/deepsvg/blob/128a55244bbb7326092aff2dbb971c26de38318f/deepsvg/svglib/svg_command.py#L516...

Hi, I want to train a Residual LFQ model for audio, and this is my core code: ``` def _loss_fn(loss_fn, x_target, x_pred, cfg, padding_mask=None): if padding_mask is not None: padding_mask...

NCCL watchdog thread terminated with exception: CUDA error: an illegal memory access was encountered

1

Hi, I just want to train a small version of RWKV-V5-169m model from scratch I implement it with huggingface: ``` import torch from transformers import AutoTokenizer, AutoConfig tokenizer = AutoTokenizer.from_pretrained("RWKV/rwkv-4-169m-pile")...

Cache in different devices when use split model with dispatch_model() function and model.generate()

3

### System Info - `transformers` version: 4.40.1 - Platform: Linux-5.15.0-67-generic-x86_64-with-glibc2.31 - Python version: 3.10.4 - Huggingface_hub version: 0.20.3 - Safetensors version: 0.4.1 - Accelerate version: 0.27.0 - Accelerate config: not...

Hi, when I utilize the tensor-parallel package as the repo indicates: ``` model = transformers.LlamaForCausalLM.from_pretrained(model_path, use_flash_attention_2="flash_attention_2", torch_dtype=torch.bfloat16) tokenizer = transformers.AutoTokenizer.from_pretrained(model_path) # This is the continue pretrained LLaMA 2 7B model...

Training with unsloth

4

Currently, Unsloth can only support single GPU training, how can you implement it with 8-GPU training? Thx

How to evaluate the results?

3

Hi, this is a good work. I wonder, after obtaining the generation results, how do you evaluate the results?

Hi, when I increase the seqlen from 1024 * 8 to 1024 * 64 here: https://github.com/zhuzilin/ring-flash-attention/blob/9e2a7e543d6461cc935d44142fc99660de7b8579/benchmark/benchmark_varlen_qkvpacked_func.py#L18 Then, I run the code with ```python torchrun benchmark/benchmark_varlen_qkvpacked_func.py ``` The program starts to...