BetterChatGPT
BetterChatGPT copied to clipboard
Feature: Implement cl100k Token
Install the following: https://github.com/dqbd/tiktoken
And import:
import tiktokenModel from "@dqbd/tiktoken/encoders/cl100k_base.json"; import { init, Tiktoken } from "@dqbd/tiktoken/lite/init"; import wasm from "../../node_modules/@dqbd/tiktoken/lite/tiktoken_bg.wasm?module";