serpapi
[Google AI Overview API] AI Overview text "Fully empty"
Valid requests with a page_token coming back empty despite being within the 4 minute window:
Initial request:
Follow-up:
This was initially reported by a customer (see search inspect links below) but their initial search also threw up the following:
Public links: Playground 1 | Playground 2 | Documentation
Internal links: Intercom | Search Inspect 1 | Search Inspect 2
A high-volume customer reported internal server errors from this engine, which I believe is related.
We have released a fix which has resolved all the related issues with this engine.
User from the above comment reached back out that the issue is still happening. The AI Overview request was about 2 minutes after the initial request. Tested the search again and it did seem to work when the AI Overview request was sent immediately after the initial request. If the new threshold is 2 minutes then can we update the documentation?
Initial Search | AI Overview Search Intercom
I have investigated further about delays in between requests. From my testing I have found that google has changed expiration time of these tokens. Page_token is now valid for only 1 min after the Initial Search is made. Since these tokens are generated from google side, we have no control over its expiration time. I will open a separate issue to update our docs.
Just to be on safe side, try to use these tokens within 45 secs.
| Delay (seconds) | Success Rate | Requests | Percentage |
|---|---|---|---|
| 120 | 0/10 | 10 | 0.0% |
| 60 | 0/9 | 9 | 0.0% |
| 59 | 0/10 | 10 | 0.0% |
| 55 | 10/10 | 10 | 100.0% |
| 45 | 10/10 | 10 | 100.0% |
| 30 | 9/9 | 9 | 100.0% |
I also tested requests with delay of less than 2 sec for more 1000 reqs, those queries worked perfectly with no errors:
Overall Success Rate: 1000/1000 (100.0%)
Average Time Taken: 2.96s