Tokenizer
Tokenizer copied to clipboard
Tokenizer TS Web Browser Compatability
Hello,
The TS package doesnt work on web browser due to its usage of fs and path, is there any plans to support web browsers?
Cheers!
Sorry, web browser support is not our priority as of now, but feel free to create a PR if you really need it, the fs/path are only used to load the bpe model files, they can be replaced with http stream in browser setting.