exo icon indicating copy to clipboard operation
exo copied to clipboard
Published 1 month ago • verified publisher icon exo-explore

Get rid of transformers dependency (81.49MB)

Open AlexCheema opened this issue 1 year ago • 0 comments

  • The transformers package is ~10MB
  • We only use this for loading tokenizers
  • Tokenizers are actually very simple and we can write a minimal implementation in exo

AlexCheema avatar Oct 02 '24 13:10 AlexCheema