GoogleCloudPlatform
[Bug]: AttributeError: 'Candidate' object has no attribute 'grounding_metadata' in intro-grounding-gemini.ipynb
File Name
intro-grounding-gemini.ipynb
What happened?
For some reason the step
tool = Tool.from_google_search_retrieval(grounding.GoogleSearchRetrieval())
response = model.generate_content(PROMPT, tools=[tool])
print_grounding_response(response)
resulted in this.
Holt Skinner asked me to file this bug. I was executing the lab in a Workbench instance, not Colab. The %pip install --upgrade --user --quiet google-cloud-aiplatform didn't seem to upgrade, it outputted:
%pip install --upgrade --user --quiet google-cloud-aiplatform
WARNING: The script tb-gcp-uploader is installed in '/home/jupyter/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Note: you may need to restart the kernel to use updated packages.
pip freeze | grep ai:
aiofiles==22.1.0
aiohttp @ file:///home/conda/feedstock_root/build_artifacts/aiohttp_1710511536979/work
aiohttp-cors==0.7.0
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
aiosqlite==0.20.0
explainable-ai-sdk==1.3.3
google-cloud-aiplatform==1.51.0
google-cloud-recommendations-ai==0.7.1
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1675110562325/work
Relevant log output
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[25], line 5
1 tool = Tool.from_google_search_retrieval(grounding.GoogleSearchRetrieval())
3 response = model.generate_content(PROMPT, tools=[tool])
----> 5 print_grounding_response(response)
Cell In[24], line 5, in print_grounding_response(response)
3 print(response.candidates[0])
4 print(str(dir(response.candidates[0])))
----> 5 grounding_metadata = response.candidates[0].grounding_metadata
7 # Citation indices are in byte units
8 ENCODING = "utf-8"
AttributeError: 'Candidate' object has no attribute 'grounding_metadata'
Output of print(response.candidates[0]):
content {
role: "model"
parts {
text: "The next solar eclipse in the US will be an annular solar eclipse on October 14, 2023. This eclipse will cross the US from Oregon to Texas. \n\nAfter that, the next solar eclipse will be a total solar eclipse on April 8, 2024. This eclipse will sweep across North America from Mazatl\303\241n to Newfoundland. \n\nThe next total solar eclipse visible in the contiguous US after 2024 will be on August 23, 2044. \n"
}
}
finish_reason: STOP
safety_ratings {
category: HARM_CATEGORY_HATE_SPEECH
probability: NEGLIGIBLE
probability_score: 0.05654813349246979
severity: HARM_SEVERITY_NEGLIGIBLE
severity_score: 0.043042849749326706
}
safety_ratings {
category: HARM_CATEGORY_DANGEROUS_CONTENT
probability: NEGLIGIBLE
probability_score: 0.21667379140853882
severity: HARM_SEVERITY_NEGLIGIBLE
severity_score: 0.16940586268901825
}
safety_ratings {
category: HARM_CATEGORY_HARASSMENT
probability: NEGLIGIBLE
probability_score: 0.1710611879825592
severity: HARM_SEVERITY_NEGLIGIBLE
severity_score: 0.05399883911013603
}
safety_ratings {
category: HARM_CATEGORY_SEXUALLY_EXPLICIT
probability: NEGLIGIBLE
probability_score: 0.11338310688734055
severity: HARM_SEVERITY_NEGLIGIBLE
severity_score: 0.0699218362569809
}
grounding_metadata {
web_search_queries: "When is the next solar eclipse in the US?"
}
Output of print(str(dir(response.candidates[0]))):
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_from_gapic', '_raw_candidate', 'citation_metadata', 'content', 'finish_message', 'finish_reason', 'from_dict', 'function_calls', 'index', 'safety_ratings', 'text', 'to_dict']
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
I think there was a change to the API service in how it outputs this field.
What was weird to me is that the print showed that field the error complains about, however the dir didn't
Hey! I encountered this bug during testing as well, just wanted to ask if the problem is resolved yet, thanks!
Getting reports from @lavinigam-gcp of users running into this error in workshops with the latest client lib version of google-cloud-aiplatform (1.63.0 at the moment). I'm getting the error originally reported in this issue, whereas some users are getting a different error at the first Vertex AI Search grounding cell with:
InvalidArgument: 400 request contains an invalid argument
Did some more testing with @lavinigam-gcp and was able to pin down different paths that were leading to 3(!) different runtime errors in the notebook cell after the text on
Text generation grounded in Vertex AI Search results
Now we can add the tools keyword arg with a grounding tool of grounding.VertexAISearch() to instruct the LLM to first perform a search within your custom data store, then construct an answer based on the relevant documents
Error from latest SDK version
When using the newly released google-cloud-aiplatform==1.63.0, I get the error initially reported in this issue:
AttributeError: 'Candidate' object has no attribute 'grounding_metadata'
Workaround: Downgrade to an earlier version of the Vertex AI SDK using !pip install google-cloud-aiplatform==1.62.0
Error when creating a data store as opposed to creating an app + data store
When following the docs tutorial steps to create a website-based data store, if you inadvertently create a data store only (without an associated app), then try to run the same cell to ground on a data store, I get a 400 error (this is the primary issue that @lavinigam-gcp and team were running into):
InvalidArgument: 400 Request contains an invalid argument.
Workaround: Rather than creating a data store only, create a search app AND a data store, and it will avoid this 400 error. Be sure to use the data store ID and not the app ID in the notebook.
Error when trying to query a freshly created data store
Whether you create an app + data store or data store only as described in the previous section, we're getting a 500 error during the initial few minutes after creating and trying to query the data store by running that same "grounding with a data store":
InternalServerError: 500 Internal error encountered.
Workaround: Wait 2-5 minutes for the data store to finish, though the status in the data store doesn't show any sort of progress or status text, but I was able to reproduce that this 500 error would resolve itself after a few minutes from the creation time of the data store.
Follow up
Though these workarounds should get the grounding notebook + Vertex AI Search data store back to a working state as a quick fix, some suggested follow up actions:
- Track SDK breakage in google-cloud-aiplatform==1.63.0 and ensure a fix is in the works for future SDK versions.
- Add more explanatory text to the notebook to mention that users need to create an app + data store rather than a standalone data store (and look into why this difference is happening)
- Add more explanatory text to the notebook to mention that users might run into a 500 error while the data store is very new (and look into how to improve this error / UI status in Console)
- Consider adding client SDK code to programmatically create the app + data store to minimize errors when setting up the app + data store in the console