[Google Search API] `num` Ignored for Some Searches

[schaferyan]

A customer reported the num parameter seems to be ignored for some searches. For example, when supplying 50 as the value of num, only 8 organic results are provided. This seems to mostly be happening with searches for famous individuals and entities that produce extensive knowledge graphs, i.e "Reese Witherspoon", "Bill Gates", "Led Zeppelin".

Playground | Search Inspector

[marm123]

The user reported that the issue persists. I'm marking this as urgent as it's connected to Google search results and num parameter which is used by many customers.

Intercom

Playground | Inspect Playground | Inspect Playground | Inspect Playground | Inspect Playground | Inspect

[zyc9012]

To be more precise. This happens when knowledge graph appear on the top like this

Inspect

So it also affects searches for tv series, movies, etc.

But it only affects the first page. num work on page 2 and after.

Setting the start=1 can remove the knowledge graph so num work again. The downside is the first organic result gets lost.

[ilyazub]

Related to the endless scrolling pagination (https://github.com/serpapi/public-roadmap/issues/537).

[ilyazub]

The num parameter is ignored when Google decides to respond with the continuous scroll instead of the regular pagination.

It's possible to disable continuous scroll by using an outdated user-agent. Example:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19042

(ref: https://www.google.com/search?q=Beyonce&num=100&hl=en&gl=us with an outdated user-agent)

Also related:

• https://news.ycombinator.com/item?id=36370725
• https://www.reddit.com/r/techsupport/comments/1436h13/is_there_any_way_to_turn_of_google_infinite/
• https://support.google.com/websearch/thread/191811385/how-to-turn-off-infinite-scroll-get-pagination-back-4-google-search-disability-accessibility-need?hl=en
• https://www.reddit.com/r/google/comments/zolrif/turn_off_infinite_scroll/

It should be possible to disable the continuous scroll in settings.

(ref: https://www.google.com/preferences?hl=en&fg=1#tabVal=1)

But it only adds the "See More" button instead.

[schaferyan]

@ilyazub could using an outdated user agent have other possibly undesired effects, such as omitting certain new elements from searches?

[ilyazub]

@ilyazub could using an outdated user agent have other possibly undesired effects, such as omitting certain new elements from searches?

Unfortunately, yes — lots of new SERP features would be missing in that case. Even using a work-around with start=1 reduces the number of SERP features.

[schaferyan]

But it only affects the first page. num work on page 2 and after.

Setting the start=1 can remove the knowledge graph so num work again. The downside is the first organic result gets lost.

My understanding is that it's already not advisable to use the num parameter if you want to get the knowledge_graph or other special elements. We already advise our customers that using num may cause Knowledge Graph elements not to appear.

Perhaps we could apply start =1 if num!=10 and just make it explicit in our documentation that using num will not return the knowledge_graph?

[ilyazub]

Perhaps we could apply start =1 if num!=10 and just make it explicit in our documentation that using num will not return the knowledge_graph?

This workaround is best done on the client side, not on the SerpApi. I'm not sure all SerpApi users want this and it totally depends on the Google Search output. (Similar to https://github.com/serpapi/public-roadmap/issues/658.)

[schaferyan]

Another customer with this issue:

Intercom

[aciddjus]

We could also use gbv=1 to retrieve no-js version of the page. This would mainly be for users interested in only organic results.

No-js version of the page has normal pagination, and num and start parameters work as expected.

This would mean that we would have to support no-js parser.

[schaferyan]

Another higher volume customer reported this:

Intercom

[schaferyan]

Another high volume customer reported this:

Intercom

[aciddjus]

While we are still exploring the feasibility of resolving this issue, you may want to try our Google Light Search API. It contains all the most critical data, but without the extra-rich results you usually find on a regular Google Search page. Pagination works with all queries.

[hilmanski]

A user reported this issue. The user reported that using start=1 solved their issue, but of course, they'll miss the first result.

Front | Inspect

[schaferyan]

Another customer reported this issue:

Front

[Garfinkel]

A user reported this issue. The user reported that using start=1 solved their issue, but of course, they'll miss the first result.

Would a workaround be to count the # of results returned, and if less than expected run it again with start=1 and then combine and dedupe the results? Not ideal but that should work, right? Looking for a short-term solution while your team thinks about a long-term implementation.

[hilmanski]

Hi @Garfinkel

Currently, we recommend trying Google Light API as an alternative solution. This is a different API for getting Google search results page. Documentation: https://serpapi.com/google-light-api Example: https://serpapi.com/playground?engine=google_light&q=coffee&gl=us&hl=en&num=100

Our customer mentioned that using start=1 parameter and num=100 can get all the 100 results (but missing the first result) or you can also try start=0 to still get the first result. Example in our playground

We apologize for this issue, and thank you for your patience.