Joe Cummings

If the bottleneck is really in the teardown (setup is unavoidable I believe), then we could opt for a similar approach to HF: ```python batch = batch.to(device) with torch.no_grad(): ......

@Harthi7 Would you mind merging main branch and running the linter? Then, we can go ahead and get this merged :)

Hey @kailashg26 - good question! The error here comes from the fact that the model utilizes a fused QKV while torchtune does not natively support this. You should take a...

> Hi [@joecummings](https://github.com/joecummings), Do you think is this the correct way to do it? > > First, I create a convert weights file in torchtune/models/llama2/llama2_qkv.py with > > ``` >...

> Also, [@joecummings](https://github.com/joecummings) could you give some insights into how I should input the dataset, which has input IDs and labels? Can you provide an example? By input IDs, do...

> Hi [@joecummings](https://github.com/joecummings) , so I get some error when running llama-270B-qkv > > **llama2_qkv.py code in torchtune/models/llama2/** > > ``` > from typing import Dict, Optional > import torch...

> Hi [@joecummings](https://github.com/joecummings) , this is the dataset [huggingface.co/datasets/regisss/scrolls_gov_report_preprocessed_mlperf_2](https://huggingface.co/datasets/regisss/scrolls_gov_report_preprocessed_mlperf_2) I'm trying to use. Looks like this format is not supported right? Could you let me know how to use this...

> Hi @zhangtemplar , you're changing the generic `convert_weights` function. Qwen2.5 already has a specific convert weights function [here](https://github.com/pytorch/torchtune/blob/main/torchtune/models/qwen2/_convert_weights.py?rgh-link-date=2025-01-08T03%3A15%3A36Z) which handles the biases of the linear projections. > > In...

No actual bug here - closing.

Great catch! We'd definitely welcome a small PR for this if you want to do it, otherwise we can try to get to it soon.