public-roadmap
public-roadmap copied to clipboard
serpapi
[Google Search API] `total_results` missing in the response
A customer reported that total_results is missing in the response from the Google Search API.
I was not able to reproduce it in the Playground, as total_results was always present in the response when executing requests with the same parameters as the customer.
@martin-serpapi
Could you please try to reproduce it? I've tried many times and tried to investigate the inspection you shared but no luck.
I think it's something related to the proxy, the connection was cut or lost during the loading of the page, since I can see that the HTML in the inspect is not complete.
total_results is missing from actual Google results. It's not something we are going to be able to fix.
I'm following up on this as another user reported the issue. It seems that the total_results are sometimes returned and sometimes not for the exact same query. Is this Google's behavior? Here are examples:
No total_results:
The same query was performed within one minute of the one above with the same parameters:
I want to make sure the issue is on Google's side, as I was constantly getting results with and without total_results via our API, but I wasn't able to reproduce it via regular Google search, as every time I tried to search something, the total_results were returned at the top of the SERP.
Playground | Inspect - no total_results | Inspect - total_results scraped
Intercom
A user reports a similar case.
As an alternative, we can get the total_results by visiting the last page as discussed here: https://github.com/serpapi/public-roadmap/issues/1196 https://github.com/serpapi/public-roadmap/issues/1316
Following up on this as one of our Enterprise Customers is concerned about the fact that there are no total_results in some requests. Can we double-check if this is an issue on Google's side, given my previous message? I couldn't get a response directly from Google with no information about total_results after many retries. It could be just a coincidence, but I want to make sure of that.
Marking it as prioritized since it's a request from our Enterprise Customer.
The presence of total_results can be affected by the cookie NID, for example
Cookie not returning total_results:
NID=513=O9MsFl2a2Ql03gtwqP-2v4rUfAr8zkze_7WbzWUIHX1JL4iBjthtHAird7ufyFoZN5V5P1AQzGhuAO8KVhVpB2jrWLf7Fy23LpVAs3eM4GlyfjM12mPgj9BoPFqAfAThyDWHnZ_6FBxC0TMS7SNdEbnJvc2BwzyOwRjUeDAk3CEK4nAAFhbtq75bmXk7
Cookie returning total_results: NID=513=VYWyUMudUUy943Z627NZrW42IGxv7FrkuTzpo5J7i041W14vu7MpRXOFs8muiN1SEsm0zhYwoAxhXxYSIgMwvEq22iXOBweM-6hGlV0XW-jorLfYWsgByA-1eq94fe11SDhpZSb0mKEmyMLN7u0c8Mk9Y1pEZlssg6J1Z6rTNgE
Thank you, @zyc9012, for investigating that! I'll let the customer know and close the issue now, as we don't want to force specific cookies just to get total_results, as there might be some side effects on the API.