torchtune icon indicating copy to clipboard operation
torchtune copied to clipboard
verified publisher icon pytorch

Bidirectional truncation in Llama4.

Open krammnic opened this issue 9 months ago • 4 comments

Really simple, just add argument that is already supported.

krammnic avatar May 15 '25 08:05 krammnic

Would you like to open this up to community contributions?

joecummings avatar May 15 '25 17:05 joecummings

Sure!

Would you like to open this up to community contributions?

krammnic avatar May 15 '25 20:05 krammnic

Can you provide a few more details on how this could be implemented then? Put some acceptance criteria and code pointers?

joecummings avatar May 15 '25 20:05 joecummings

@joecummings Sure.

Task

We have a special argument truncation_type which is the passed in truncate:

https://github.com/pytorch/torchtune/blob/e5ee1b2fcd25a411a4d0889849c1528189d56616/torchtune/models/llama3/_tokenizer.py#L341

Unfortunately, it is not supported by llama4 tokenizer! The task is quite simple: add support of the truncation_type similarly how it is done in other models.

Acception criteria

Test are passing and few sanity check in order to check the correctness.

krammnic avatar May 16 '25 17:05 krammnic

I can take this!

adheep04 avatar Jun 05 '25 21:06 adheep04

@adheep04 LET YOU PR!

krammnic avatar Jun 06 '25 09:06 krammnic