llama.cpp
llama.cpp copied to clipboard
Add --pre-tokenizer option to convert
Allow for overwriting pre-tokenizer when converting model. It's useful when models require third-party code to run (i.e MiniCPM-V-2.5).
@cmp-nct This will interest you