gpt_bpe
gpt_bpe copied to clipboard
GLM-4.5 support
I had to move specific logic about large vocab sizes and how we're tokenizing spaces for GLM into specific "feature flags" in special_config.json