tokenisation topic

TokenScript schema, specs and paper

SmartTokenLabs

Open morphology for Finnish

flammie

A Python 3 module that provides functions for splitting identifiers found in source code files.

casics

Frames Android: making native card payments simple

checkout

Frames iOS: making native card payments simple

checkout

Ungreedy subword tokenizer and vocabulary trainer for Python, Go & Javascript

alasdairforsythe