deepgram
Unexpectedly reached token limit for keyterms with nova-3 model
What is the current behavior?
For a request with keyterms, I get the following error:
DeepgramApiError: DeepgramApiError: Keyterm limit exceeded. The maximum number of tokens across all keyterms is 500. (Status: 400)
Steps to reproduce
The log gives me the following raw request to reproduce this behavior:
INFO:httpx:HTTP Request: POST https://api.deepgram.com/v1/listen?keyterm=Air&keyterm=Traffic&keyterm=Control&keyterm=Communications&keyterm=LKPR&keyterm=Praha&keyterm=Ruzyne&keyterm=Radar&keyterm=AKEVA&keyterm=ARVEG&keyterm=BAGRU&keyterm=BAROX&keyterm=BAVIN&keyterm=BEKVI&keyterm=ELMEK&keyterm=ELPON&keyterm=ERASU&keyterm=EVEMI&keyterm=KENOK&keyterm=KUVIX&keyterm=LETNA&keyterm=RATEV&keyterm=RISUK&keyterm=SOMIS&keyterm=SULOV&keyterm=TIPRU&keyterm=UTORO&keyterm=OKLBA&keyterm=PGT302&keyterm=EZS93AV&keyterm=WZZ9654&keyterm=TAP1244&keyterm=QTR8005&keyterm=PGT1527&keyterm=DHK547&keyterm=RYR73AH&keyterm=DLH6YW&keyterm=OKOUU37&keyterm=AFR10BE&keyterm=RYR2YK&keyterm=SDR4272&keyterm=OKLLZ&keyterm=TRA6868&keyterm=UAE139&keyterm=OKDKB&keyterm=CSA2DZ&keyterm=RRR4941&keyterm=DEBLN&keyterm=OKPMP&keyterm=LZB301&keyterm=BRU862&keyterm=PGT76A&keyterm=OKMHZ&keyterm=RYR2122&keyterm=DLH4RM&keyterm=OKAVK&keyterm=OKWYZ&keyterm=BLA1RK&keyterm=OKHBT&keyterm=CSA6KG&keyterm=OKXAI04&keyterm=AFR108Z&keyterm=OKYAI14&keyterm=RYR5XL&keyterm=UAE140&keyterm=OKUUR14&keyterm=OKKEA&keyterm=EWG7AB&keyterm=EWG7677&keyterm=CSA94D&keyterm=OKSOL&keyterm=OKHSI&keyterm=SFS80&keyterm=OKTOP&keyterm=&keyterm=&keyterm=Oklahoma&keyterm=Alfa&keyterm=Sun&keyterm=Turk&keyterm=Topswiss&keyterm=Wizzair&keyterm=Air&keyterm=Portugal&keyterm=Qatari&keyterm=World&keyterm=Express&keyterm=Ryan&keyterm=Lufthansa&keyterm=France&keyterm=Sundair&keyterm=Transavia&keyterm=Emirates&keyterm=Csa&keyterm=%2F&keyterm=Czech&keyterm=Ascot%2Fkittyhawk&keyterm=Debonair&keyterm=Okapi&keyterm=Flying&keyterm=Bulgaria&keyterm=Belarus&keyterm=Avia&keyterm=Okayjet&keyterm=All&keyterm=Charter&keyterm=Eurowings&keyterm=Topo&keyterm=Soko&keyterm=alpha&keyterm=bravo&keyterm=charlie&keyterm=delta&keyterm=echo&keyterm=foxtrot&keyterm=golf&keyterm=hotel&keyterm=india&keyterm=juliett&keyterm=kilo&keyterm=lima&keyterm=mike&keyterm=november&keyterm=oscar&keyterm=papa&keyterm=quebec&keyterm=romeo&keyterm=sierra&keyterm=tango&keyterm=uniform&keyterm=victor&keyterm=whiskey&keyterm=xray&keyterm=yankee&keyterm=zulu&keyterm=climb&keyterm=climbing&keyterm=descend&keyterm=descending&keyterm=passing&keyterm=feet&keyterm=knots&keyterm=degrees&keyterm=direct&keyterm=maintain&keyterm=identified&keyterm=ILS&keyterm=VFR&keyterm=IFR&keyterm=contact&keyterm=frequency&keyterm=turn&keyterm=right&keyterm=left&keyterm=heading&keyterm=altitude&keyterm=flight&keyterm=level&keyterm=cleared&keyterm=squawk&keyterm=approach&keyterm=runway&keyterm=established&keyterm=report&keyterm=affirm&keyterm=negative&keyterm=wilco&keyterm=roger&keyterm=radio&keyterm=radar&model=nova-3&smart_format=true "HTTP/1.1 400 Bad Request"
Expected behavior
At first glance, I'd consider a "keyterm" to correspond to one "token". Since there are 170 keyterms in this request, there should be no limit error.
So, either my assumption is false or the server-side implementation does the counting wrong.
Unfortunately, it is not clear from the documentation what exactly is meant by a "token". As this term is also used in pricing and billing, it would be very important to clarify this to help customers make an informed decision.
Please tell us about your environment
$ lsb_release -a
LSB Version: :core-5.0-amd64:core-5.0-noarch
Distributor ID: Fedora
Description: Fedora release 41 (Forty One)
Release: 41
Codename: FortyOne
$ uname -a
Linux gallifrey 6.12.6-200.fc41.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Dec 19 21:06:34 UTC 2024 x86_64 GNU/Linux
$ python --version
Python 3.13.2
$ python -c "import deepgram; print(deepgram.__version__)"
v3.10.0
Hey @sfo ! Great question.
The term "token" is used in the context of AI models - Deepgram's AI glossary has an entry on tokenization that can give some background on the subject.
;tldr there isn't a 1:1 mapping between between word count and token count. As a super rough hueristic, the token count will be closer to the syllable count.
While we don't have the exact tokenizer used by Nova-3 available, we have another tokenizer that is publicly available at https://tokenizer.deepgram.com/ that can give you a rough estimate for the number of tokens in your particular query. For example, I plugged in Communications Praha LKPR (three of your keyterms), and it told me that was 6 tokens total.
Any plans to increase the key term limit for nova-3?
Unfortunately, this is blocking our company from upgrading from nova-2 to nova-3.
The keywords parameter in nova-3 seems to accept at least 3x the number of tokens. (I'm not sure what the upper limit is)
