generative-ai-python icon indicating copy to clipboard operation
generative-ai-python copied to clipboard

`count_tokens` should allow empty content

Open markmcd opened this issue 9 months ago • 2 comments

Description of the bug:

Currently, this code:

model = genai.GenerativeModel(model_name, system_instruction="Talk in rhymes")
model.count_tokens("")

Throws this error:

TypeError: contents must not be empty

It is possible to work around it by passing contents={'parts': {'text': ''}} but this requires some mental gymnastics to derive and to understand.

Now that we have system instructions and other count-able metadata outside of content, we should allow counting that metadata in the SDK. One way would be to allow empty content, but only when used through count_tokens.

markmcd avatar May 08 '24 08:05 markmcd

yeah...protos mark contents as optional when passed individually, but is a required field when passing with the entire GenerateContentRequest. Current implementation converts passed params to GenerateContentRequest regardless.

A possible fix could be( on line 338 ):

if not contents:
    contents = [""]

https://github.com/google-gemini/generative-ai-python/blob/bb58ab83865a4a2f60688af7d95166bfd379a72a/google/generativeai/generative_models.py#L323-L340

Not sure, if this is a good fix. (will it increase the token count, as a blank token is also being passed to the req?)

mayureshagashe2105 avatar May 10 '24 10:05 mayureshagashe2105

I'm not sure it needs anything other than: https://github.com/google-gemini/generative-ai-python/pull/342

MarkDaoust avatar May 14 '24 21:05 MarkDaoust