Can't seem to get it running

[Egregius]

Hi,

I'd like to use your script for exporting my Google timeline. My previous script stopped working several weeks ago. Unfortunately your script also drops errors. Do you still use this script? Do you have a fix for it?

root@www:/TimelineExtractor/src# python3 extract.py -l debug -c cookie date 2022-08-08

INFO: Calculating location history for 1 date(s)
DEBUG: Starting new HTTPS connection (1): www.google.com:443
DEBUG: https://www.google.com:443 "GET /maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2022!2i7!3i8!2m3!1i2022!2i7!3i8 HTTP/1.1" 302 303
DEBUG: Starting new HTTPS connection (1): timeline.google.com:443
DEBUG: https://timeline.google.com:443 "GET /maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2022!2i7!3i8!2m3!1i2022!2i7!3i8 HTTP/1.1" 302 489
DEBUG: Starting new HTTPS connection (1): accounts.google.com:443
DEBUG: https://accounts.google.com:443 "GET /ServiceLogin?continue=https://timeline.google.com/maps/timeline/kml?authuser%3D0%26pb%3D!1m8!1m3!1i2022!2i7!3i8!2m3!1i2022!2i7!3i8&rart=ANgoxcccA8GoKkOtWE-u5pn8bQVRe6KcXxm_uLrsTuoSWitCUvgGH7eiu6vKmVYP9OhlkHVvLhmsvd5DExQCfRu5XPWHxZVrbQ&osid=1&authuser=0 HTTP/1.1" 200 None
Traceback (most recent call last):
  File "/sql/TimelineExtractor/src/extract.py", line 89, in <module>
    main()
  File "/sql/TimelineExtractor/src/extract.py", line 77, in main
    History = GetLocationHistoryForDates(Args.date, AuthCookie)
  File "/sql/TimelineExtractor/src/extract.py", line 22, in GetLocationHistoryForDates
    return LocationHistory.GetDates(Dates, AuthCookie)
  File "/sql/TimelineExtractor/src/LocationHistory.py", line 79, in GetDates
    LocationHistory = GetDate(SortedDates[0], AuthCookie)
  File "/sql/TimelineExtractor/src/LocationHistory.py", line 67, in GetDate
    return ET.ElementTree(ET.fromstring(Response.text))
  File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 1347, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: mismatched tag: line 1256, column 4

[Stadly]

Hello @Egregius,

Sorry for the extremely late response! It seems that you are not able to authenticate. Have you stored your authentication cookie?

[Egregius]

Yes, I did. I now found in the other issue the debug option. This is the output:

INFO: Calculating location history for 1 date(s)
DEBUG: Starting new HTTPS connection (1): www.google.com:443
DEBUG: https://www.google.com:443 "GET /maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2023!2i3!3i16!2m3!1i2023!2i3!3i16 HTTP/1.1" 302 305
DEBUG: Starting new HTTPS connection (1): timeline.google.com:443
DEBUG: https://timeline.google.com:443 "GET /maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2023!2i3!3i16!2m3!1i2023!2i3!3i16 HTTP/1.1" 302 491
DEBUG: Starting new HTTPS connection (1): accounts.google.com:443
DEBUG: https://accounts.google.com:443 "GET /ServiceLogin?continue=https://timeline.google.com/maps/timeline/kml?authuser%3D0%26pb%3D!1m8!1m3!1i2023!2i3!3i16!2m3!1i2023!2i3!3i16&rart=ANgoxccUFIr2YzYwuAWGenGBW-jWnLlyHPrOppg5NaMuDFRFn-T1Dpn_TLGbLX0VS-f2TSDpgrRrfYqO9IYwqQhVdGOHQf-z7g&osid=1&authuser=0 HTTP/1.1" 302 None
DEBUG: https://accounts.google.com:443 "GET /InteractiveLogin?authuser=0&continue=https://timeline.google.com/maps/timeline/kml?authuser%3D0%26pb%3D!1m8!1m3!1i2023!2i3!3i16!2m3!1i2023!2i3!3i16&osid=1&rart=ANgoxccUFIr2YzYwuAWGenGBW-jWnLlyHPrOppg5NaMuDFRFn-T1Dpn_TLGbLX0VS-f2TSDpgrRrfYqO9IYwqQhVdGOHQf-z7g&ifkv=AQMjQ7S7S3a9Wya0nKpkEjic7MU7QQ4i0TrdF2ukN-CCuM-1Vh9aEV-DB9RIkB_1o43YOfz3nOOC8g HTTP/1.1" 302 698
DEBUG: https://accounts.google.com:443 "GET /v3/signin/identifier?dsh=S-1727306228%3A1681886783374117&authuser=0&continue=https%3A%2F%2Ftimeline.google.com%2Fmaps%2Ftimeline%2Fkml%3Fauthuser%3D0%26pb%3D%211m8%211m3%211i2023%212i3%213i16%212m3%211i2023%212i3%213i16&ifkv=AQMjQ7RQWsTcFj2dy_EHYDwACtRvS_UQigW0esHKzv22lF9agXP9PIdKcymlZnRcJzT40dhHvZ7Apg&osid=1&rart=ANgoxccUFIr2YzYwuQWGenGBW-jWnLlyHPrOppg5NaMuDFRFn-T1Dpn_TLGbLX0VS-f2TSDpgrRrfYqO9IYwqAhVdGOHQf-z7g&flowName=WebLiteSignIn&flowEntry=ServiceLogin HTTP/1.1" 200 None
Traceback (most recent call last):
  File "/sql/TimelineExtractor/src/extract.py", line 89, in <module>
    main()
  File "/sql/TimelineExtractor/src/extract.py", line 77, in main
    History = GetLocationHistoryForDates(Args.date, AuthCookie)
  File "/sql/TimelineExtractor/src/extract.py", line 22, in GetLocationHistoryForDates
    return LocationHistory.GetDates(Dates, AuthCookie)
  File "/sql/TimelineExtractor/src/LocationHistory.py", line 79, in GetDates
    LocationHistory = GetDate(SortedDates[0], AuthCookie)
  File "/sql/TimelineExtractor/src/LocationHistory.py", line 67, in GetDate
    return ET.ElementTree(ET.fromstring(Response.text))
  File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 1347, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: syntax error: line 1, column 0

I can add to this that I had a script running for several months but it stopped working. I guess Google changed something in the authentication. Are you still using this script?

[Egregius]

Strange thing is that there are multiple requests:

[Stadly]

I haven't used the script in quite a while. Tried it now and couldn't get it to work. It definitely seems like Google changed something. I'll see if I can find a fix for it, but it might take some time before I can look into it.

[Egregius]

OK, good luck!

[Egregius]

Thanks for the fix! Finally got it working, so I don't have to export every day my timeline anymore :) Hope the rapt and cookie will last long time so I don't need to update them regularly, any ideas on that?

[Stadly]

Great to hear! I don't have any experience with the new authentication yet. Do you have any insights, @GRWalter?

[Egregius]

Mine just already expired, had to copy the rapt and cookie again. Maybe I should copy them from a private window? Or don't use the timeline anymore on my computer?

[GRWalter]

I think the cookie might last a while, but the rapt seems to expire pretty quickly, though I don't know exactly how long it lasts. Copying from a private window probably won't affect anything.

Google will be removing the ability to access the Maps timeline from the computer in November of this year (specifically Nov. 19th), so just be sure to download your data before then. Beware, though, that using this script might flag you as a bot and require you to fill out captchas in the browser from time to time.

After Nov. 19, timeline data will only be stored locally on your phone. The warning that is shown in the Google Maps app says that you need to switch to local storage by Nov. 19 to prevent losing timeline history.

From what I can tell, once you switch to local storage, you can only download timeline data in JSON format. If that JSON file is the same as the one created from Google Takeout, then it will contain significantly less data than the KML files you can download right now. For example, a route with 100 logged coordinates in the KML file might only show 3 logged coordinates in the JSON file.

[Stadly]

Oh! This is new to me. I don't think I have run into such a warning on my phone. Can it be region specific?

[Egregius]

OMG, I hope that's not true. I've been saving my Google History in a database to plot it on a map. It would be a great shame if this were no longer possible. I also haven't seen that warning yet (in Belgium). If it's no longer possible I'll have to look for another app that can do the same. Google Timeline on the other hand is extremely useful because it automatically maps destinations from Google Maps.

[GRWalter]

It's possible; I'm not sure on that. I'm in the US, though. Here's Google's blog post about it. Looks like the date is variable, though; for me, it's Nov. 19, but that could be different for others. I only see the notice when trying to view my timeline in the app; I don't see the notice on the main screen.