Is also the Google Maps information is in Protobuf

[MusiCode1]

Google Maps has a very strange format to convey the information to the customer. This is a lot of nested arrays, which contain a lot of NULL values. It looks like this:

{"c":0,"d":")]}'\n[[\"white house\",[[null,null,null,null,null,null,null,null,\"L1m9X6rhA5CKlwSauKDQBw\",\"0ahUKEwjqtNnQ75vtAhUQxYUKHRocCHoQmBkIAigA\",[\"white house\",0]\n]\n]\n,null,1,null,[\"1606244655560\"]\n,null,null,null,null,null,0]\n,[[5.345248603008451E7,-20.522437649999997,35.14462475]\n,[0.0,0.0,0.0]\n,[1024,768]\n,13.1]\n,null,null,null,null,null,\"L1m9X6rhA5CKlwSauKDQBw\",null,[]\n,[[[\"m\",[3,0,0]\n,8,[534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534247556,534247556,534255710,534255710,534254714,534254714,534227504,534240512,534240512,534240512,534253622,534254714]\n]\n,[\"m\",[2,0,0]\n,4,[534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255614,534255710,534255710,534255710]\n]\n,[\"m\",[4,3,1]\n,8,[534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710,534255710]\n]\n]\n]\n,null,null,null,null,null,[[[2,\"spotlight\",null,null,null,null,null,[null,null,null,null,null,null,null,null,null,null,null,[\"white house\",[\"112308031950842969741\",\"\"]\n,[[null,null,\"0x0:0xb3e6e80a69d78dc1\"]\n,[null,null,\"0x0:0x67ae4478a534c4d6\"]\n,[null,null,\"0x0:0xc83d885f202b2ecf\"]\n,[null,null,\"0x0:0x424daceea73a1f68\"]\n,[null,null,\"0x0:0xe13ae4daf2eec66b\"]\n,[null,null,\"0x0:0xf5a81e78fa2e0d78\"]\n,[null,null,\"0x0:0x53fd99e4f5b8c7f\"]\n,[null,null,\"0x0:0x1d89849223d7f90d\"]\n,[null,null,\"0x0:0xc5edb6b6c28d4154\"]\n,[null,null,\"0x0:0x62ff5e6af80250a0\"]\n,[null,null,\"0x0:0x30e1c1aa4416e2c0\"]\n,[null,null,\"0x0:0x713dc1909209bc1a\"]\n,[null,null,\"0x0:0x50d2a43e03c42a73\"]\n,[null,null,\"0x0:0x5860d8d32225a85\"]\n,[null,null,\"0x0:0xd0b2a5a1da50756e\"]\n,[null,null,\"0x0:0x715969d86d0b76bf\"]\n,[null,null,\"0x0:0xe48589933a9c28e1\"]\n,[null,null,\"0x0:0x30162385ca0fbd7d\"]\n,[null,null,\"0x0:0x95ca0169317472f2\"]\n,[null,null,\"0x0:0x3e49841d60cf0fec\"]\n]\n,null,null,null,null,null,null,0]\n,[null,\"a\",null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,[null,null,null,null,0,0,null,null,1,null,null,1,null,null,null,0,null,null,null,1,1]\n,null,null,null,[null,null,null,null,null,2,3,2]\n]\n,null,null,null,null,null,[12,14,29,30,61,70]\n,null,null,null,null,null,[null,null,null,2]\n]\n]\n]\n,[[52]\n]\n]\n,null,null,null,null,[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,\"IAE\\u003d\"]\n,null,null,null,null,null,[4]\n,null,null,null,null,null,null,null,\"Q2dBd0Fn\",null,null,0,[\"white house\"]\n]\n","e":"L1m9X6rhA5CKlwSauKDQBw","p":true,"u":"https:\/\/www.google.com\/search?tbm=map\u0026hl=en\u0026q=white%20house\u0026tch=1\u0026ech=1"}/*""*/

Sample link: https://www.google.com/search?tbm=map&hl=en&q=white%20house&tch=1&ech=1

Is it also Protobuf? And how do you decipher it?

[marin-m]

Hello,

Yes, this is Protobuf serialized to JavaScript. The produced JSON array indexes just correspond to the field numbers of the original Protobuf message, and null values are inserted when they are gaps in the field numbers numbering and field numbers aren't just linear, or some fields are not used.

This serialization method is very much used in Google products, including in Google Search and other parts.

AFAIK, there is no way to extract the original serialization structures because the Closure Compiler-transformed code that reads this usually indexes the generated JSON array fields directly by their indexes.

However, by luck, it is not impossible that the fields of this format may match other messages used with other Protobuf-based serialization method (e.g, binary Protobuf sent by a mobile application) that may use similar message structures.

Regards,

[ilyazub]

@MusiCode1, you can extract data from it by accessing the nested arrays. According to elm-lang creator, Google never removes existing fields, but I haven't found an official statement about it. It works pretty stably for us for more than a year. I can't share the code of the parser but we have a paid API.