serpapi
[Yelp Place API] Can't scrape a place: HAtz-IYcrY7rhD61TpTZqg
A customer reported an issue when scraping a place.
Place ID: HAtz-IYcrY7rhD61TpTZqg or midlands-landscape-and-lawn-lexington
This is a valid place on Yelp: https://www.yelp.com/biz/HAtz-IYcrY7rhD61TpTZqg
Playground | Inspect | Front
Related to: https://github.com/serpapi/public-roadmap/issues/1633 https://github.com/serpapi/SerpApi/issues/4830
Another related issue: https://github.com/serpapi/public-roadmap/issues/1241
The customer who reported this reached out with a few additional examples of searches that failed using the Yelp Search API specifically.
I've opened that issue here: #1838
Sharing some findings.
Yelp uses Hypernova, which works like NextJS Server Side.
Majority of the time, this is how the key should looks like. In this case, data is embedded in the HTML.
But for some reason, it doesn't work for some place ID. When ...not_use is present, the data need extra request (GraphQL) to be retrieved. This is pretty consistent, so retrying won't work.
One way to make this work is just use the GraphQL endpoint, it should works for all cases.
curl 'https://www.yelp.com/gql/batch' \
--data-raw '[{"operationName":"GetBusinessesOwnedByCurrentConsumerUser","variables":{"guv":"B7ABF41C1BC104EB","minConfidenceLevel":"HIGH_CONFIDENCE"},"extensions":{"operationType":"query","documentId":"2c212cc7349b7a3fde84aa37dfeea7538d18b64479daed69722ed9a9c00b77a8"}},{"operationName":"GetConsumerHeaderData","variables":{"isBizEncIdUndefinedOrEmpty":false,"locationQueryParam":null,"logoVariant":"DEFAULT","bizEncId":"HAtz-IYcrY7rhD61TpTZqg"},"extensions":{"operationType":"query","documentId":"65b496c270ca0f001cad72346f4bad828e3774be860d97882bfcc4cec69d8e28"}},{"operationName":"GetSpecialHoursAlert","variables":{"BizEncId":"HAtz-IYcrY7rhD61TpTZqg"},"extensions":{"operationType":"query","documentId":"607f1448c157570b0f9ace5547e8e0949cc2eabb6d1e5aa0d6275e25a8e2495a"}},{"operationName":"GetBizHeaderData","variables":{"BizEncId":"HAtz-IYcrY7rhD61TpTZqg","BizFreshnessIsEnabled":true,"IncludeClaimability":false,"RoundingMethod":"NEAREST_TENTH"},"extensions":{"operationType":"query","documentId":"77b8664e7e28d608aad0c6394ee33a934a8952f4a80d952e48f1688d0dd6cd90"}},{"operationName":"ActionBarControllerData","variables":{"businessEncid":"HAtz-IYcrY7rhD61TpTZqg","searchForFriendsUrl":"/send_to_friend/friend_type_ahead_v2","createCollectionUrl":"/collection/user/new","addCollectionItemUrl":"/collection/user/item/add","removeCollectionItemUrl":"/collection/user/item/remove","connectFollowUrl":"/yelp_connect/biz/HAtz-IYcrY7rhD61TpTZqg/follow"},"extensions":{"operationType":"query","documentId":"a47440169ea41ad5c6ffcf52180339a9c4be75d907a959cf0fb4725833cfc4a1"}},{"operationName":"GetBusinessHealthScoreAlert","variables":{"BizEncId":"HAtz-IYcrY7rhD61TpTZqg"},"extensions":{"operationType":"query","documentId":"9757b0dd458b4ad4a5922f28cbd77a1d284093c6b0e84a13c95f01c56f776e9d"}},{"operationName":"GetVerifiedLicenseData","variables":{"BizEncId":"HAtz-IYcrY7rhD61TpTZqg"},"extensions":{"operationType":"query","documentId":"0be913fb52b79279a046af71b0507ce525031ad9ab30a3d9cf247bfcda54733d"}},{"operationName":"GetVibeCheckData","variables":{"BizEncId":"HAtz-IYcrY7rhD61TpTZqg"},"extensions":{"operationType":"query","documentId":"5504b1659960953d4a222d01a9d8e9c785f9dfca3d4b818dcba5e2acc90cffff"}},{"operationName":"GetBizJobsOffered","variables":{"BizEncId":"HAtz-IYcrY7rhD61TpTZqg"},"extensions":{"operationType":"query","documentId":"5e011e31923216fcc2041087c7a670c829968c77e7c39a85b5c68604e8d8fa04"}},{"operationName":"GetReviewHighlights","variables":{"encBizId":"HAtz-IYcrY7rhD61TpTZqg","locale":"en_US"},"extensions":{"operationType":"query","documentId":"e69be2b812f6a1a266ad78454831d40e9ae1949a29d960d49b7fea113acf1d97"}}]'
Another example:
https://serpapi.com/playground?engine=yelp_place&place_id=sierra-construction-services-sparks