aici icon indicating copy to clipboard operation
aici copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

warning when unusual tokenization detected in output

Open mmoskal opened this issue 1 year ago • 0 comments

turn LLM output to string, encode and compare token ids

mmoskal avatar Oct 17 '23 00:10 mmoskal