FAST: question about state

[xianjiedai]

Hello,

Thanks for the excellent work!

@kpertsch: I am curious about the processing of state in FAST related experiments: prompt is concatenated with discretized state tokens to form a new string, such as "Task: pick up the block, State: 25 116 191 128;\n". I am wondering whether it's approriate to use FAST tokenizer to encode state as well instead of using Paligemma tokenzier. I would appreciate it a lot if you could share some insights in this decision.

Thanks in advance for your time and answer!

Cheers, Xianjie Dai