garmin-connect-export icon indicating copy to clipboard operation
garmin-connect-export copied to clipboard

Unexpected HTTP error (503)

Open tymmej opened this issue 7 years ago • 28 comments

Hi,

I'm using latest version of script. I get raise Exception('Failed. Got an unexpected HTTP error (' + str(e.code) + ').') Exception: Failed. Got an unexpected HTTP error (503). when trying to run script. Last week everything was working.

I tried putting url (url_gc_gpx_activity + a['activity']['activityId'] + '?full=true) and I get The requested endpoint is retired

URL for downloading through browser is https://connect.garmin.com/modern/proxy/download-service/export/gpx/activity/12691xxxxxx

tymmej avatar Jul 24 '16 17:07 tymmej

Uh oh! The URL you provided works for me (replacing the xxxxxx, of course). To clarify, does it work for you, too?

kjkjava avatar Jul 25 '16 03:07 kjkjava

The url used by script (http://connect.garmin.com/proxy/activity-service-1.1/gpx/activity/) does not work.

tymmej avatar Jul 25 '16 05:07 tymmej

@tymmej you're right. It does not work.

any idea about how to fix it?

frechina avatar Jul 26 '16 10:07 frechina

Using same url as browser? I haven't tried that and don't know if there's any difference in .gpx files.

tymmej avatar Jul 26 '16 11:07 tymmej

I tried but it does not work. it seems that some info in the cookie is needed.

Maybe @kjkjava can help us

frechina avatar Jul 26 '16 11:07 frechina

Or User-agent

tymmej avatar Jul 26 '16 14:07 tymmej

@tymmej @kjkjava I partially fixed the issue by:

  • Changing http://connect.garmin.com/proxy/activity-service-1.1/tcx/activity/12691xxxxxx by https://connect.garmin.com/modern/proxy/download-service/export/tcx/activity/12691xxxxxx by
  • And passing the User-agent in the GET call

However, I'm getting the following error message when getting some tcx (I don't know why) {"message":"java.util.ArrayList cannot be cast to java.util.Map","error":"ClassCastException"}

frechina avatar Jul 26 '16 16:07 frechina

The endpoint http://connect.garmin.com/proxy/activity-service-1.1/gpx/activity has been "retired" (deprecated) apparently.

If what you want is GIS coordinate data, you can get that directly as JSON using their activityDetails endpoint, https://connect.garmin.com/modern/proxy/activity-service-1.3/json/activityDetails/{activityID},

which is more efficient than downloading a GPX file and parsing it anyway.

ebrensi avatar Jul 30 '16 00:07 ebrensi

@frechina can you share patch?

tymmej avatar Jul 30 '16 03:07 tymmej

@tymmej I think @ebrensi approach is the best way to fix this. Maybe @ebrensi can share with us how to parse the activityDetails json

frechina avatar Aug 01 '16 07:08 frechina

@frechina GPX has points every second, JSON every ~40. And I want to have GPX files. It's easier to share them with others (various online visualizers, import to Strava or Endomondo...). Also in case I will resign from using Garmin Connect having GPX files will be better.

tymmej avatar Aug 01 '16 15:08 tymmej

@tymmej your comment about data point frequency is interesting. I assumed the points would be the same regardless of file format, and never bothered to look. Can someone confirm this?

I got the idea for using the json endpoint from Tapiriik, in this file https://github.com/cpfair/tapiriik/blob/master/tapiriik/services/GarminConnect/garminconnect.py

ebrensi avatar Aug 01 '16 18:08 ebrensi

@ebrensi JSON: "metrics" : [ 0.0, 51.13505808636546, 0.0, 25.0, 116.0, 0.0, 2.0000000298023223E-4, 1.469850536E12, 4.204799938201905, 14.269406605265925, 16.974758729338646, 0.0 ] }, { "metrics" : [ 1.0, 51.13502682186663, 1.0, 25.0, 116.0, 1.0, 0.0034200000762939454, 1.469850537E12, 5.202000188827515, 11.534024958488798, 16.974790580570698, 0.0 ] }, { "metrics" : [ 157.0, 51.13100937567651, 115.0, 23.0, 116.5999984741211, 157.0, 0.5354600219726563, 1.469850693E12, 19.670400810241702, 3.0502682985880014, 16.974083986133337, 63.0 ] }, { "metrics" : [ 311.0, 51.124709118157625, 269.0, 22.0, 114.5999984741211, 311.0, 1.4050400390625, 1.469850847E12, 22.406399917602542, 2.677806351428362, 16.977131897583604, 63.0 ] }, { "metrics" : [ 465.0, 51.11847021616995, 419.0, 21.0, 124.19999694824219, 465.0, 2.24947998046875, 1.469851001E12, 20.736000823974614, 2.893518404119131, 16.9703334197402, 59.0 ] }, { "metrics" : [ 619.0, 51.110126450657845, 573.0, 20.0, 118.4000015258789, 619.0, 3.2041201171875, 1.469851155E12, 22.62239971160889, 2.6522385236262296, 16.96826552040875, 0.0 ] }, { "metrics" : [ 773.0, 51.10211938619614, 727.0, 20.0, 114.4000015258789, 773.0, 4.1405400390625, 1.469851309E12, 20.45520057678223, 2.9332393875474043, 16.972166541963816, 56.0 ] }, {

JSON with ?full=true at and of url "metrics" : [ { "metrics" : [ 0.0, 51.13505808636546, 0.0, 25.0, 116.0, 0.0, 2.0000000298023223E-4, 1.469850536E12, 4.204799938201905, 14.269406605265925, 16.974758729338646, 0.0 ] }, { "metrics" : [ 1.0, 51.13502682186663, 1.0, 25.0, 116.0, 1.0, 0.0034200000762939454, 1.469850537E12, 5.202000188827515, 11.534024958488798, 16.974790580570698, 0.0 ] }, { "metrics" : [ 157.0, 51.13100937567651, 115.0, 23.0, 116.5999984741211, 157.0, 0.5354600219726563, 1.469850693E12, 19.670400810241702, 3.0502682985880014, 16.974083986133337, 63.0 ] }, { "metrics" : [ 311.0, 51.124709118157625, 269.0, 22.0, 114.5999984741211, 311.0, 1.4050400390625, 1.469850847E12, 22.406399917602542, 2.677806351428362, 16.977131897583604, 63.0 ] }, { "metrics" : [ 465.0, 51.11847021616995, 419.0, 21.0, 124.19999694824219, 465.0, 2.24947998046875, 1.469851001E12, 20.736000823974614, 2.893518404119131, 16.9703334197402, 59.0 ] }, { "metrics" : [ 619.0, 51.110126450657845, 573.0, 20.0, 118.4000015258789, 619.0, 3.2041201171875, 1.469851155E12, 22.62239971160889, 2.6522385236262296, 16.96826552040875, 0.0 ] }, {

GPX exported in browser: <trkpt lat="51.1350580863654613494873046875" lon="16.97475872933864593505859375"> <ele>116</ele> <time>2016-07-30T03:48:56.000Z</time> <extensions> <ns3:TrackPointExtension> <ns3:atemp>25.0</ns3:atemp> <ns3:cad>0</ns3:cad> </ns3:TrackPointExtension> </extensions> </trkpt> <trkpt lat="51.13502682186663150787353515625" lon="16.974790580570697784423828125"> <ele>116</ele> <time>2016-07-30T03:48:57.000Z</time> <extensions> <ns3:TrackPointExtension> <ns3:atemp>25.0</ns3:atemp> <ns3:cad>0</ns3:cad> </ns3:TrackPointExtension> </extensions> </trkpt> <trkpt lat="51.13499010913074016571044921875" lon="16.974816061556339263916015625"> <ele>116</ele> <time>2016-07-30T03:48:58.000Z</time> <extensions> <ns3:TrackPointExtension> <ns3:atemp>25.0</ns3:atemp> <ns3:cad>0</ns3:cad> </ns3:TrackPointExtension> </extensions> </trkpt> <trkpt lat="51.13496638834476470947265625" lon="16.97483357973396778106689453125"> <ele>116</ele> <time>2016-07-30T03:48:59.000Z</time> <extensions> <ns3:TrackPointExtension> <ns3:atemp>25.0</ns3:atemp> <ns3:cad>0</ns3:cad> </ns3:TrackPointExtension> </extensions> </trkpt> <trkpt lat="51.13494149409234523773193359375" lon="16.97485269047319889068603515625"> <ele>116</ele> <time>2016-07-30T03:49:00.000Z</time> <extensions> <ns3:TrackPointExtension> <ns3:atemp>25.0</ns3:atemp> <ns3:cad>0</ns3:cad> </ns3:TrackPointExtension> </extensions> </trkpt>

JSON even doesn't have info about date of activity.

Edit: ?maxSize=999999999 to get all points.

tymmej avatar Aug 01 '16 18:08 tymmej

I managed to make this work using the 1.2 API: https://connect.garmin.com/proxy/activity-service-1.2/rest.activity.activityIdValue.html

I only made a change to the tcx line (I'm importing historical data to Strava) pull request added:

diff --git a/gcexport.py b/gcexport.py index 2c6fec7..2a2bf9a 100755 --- a/gcexport.py +++ b/gcexport.py @@ -92,7 +92,7 @@ url_gc_login = 'https://sso.garmin.com/sso/login?service=https%3A%2F%2Fconne url_gc_post_auth = 'https://connect.garmin.com/post-auth/login?' url_gc_search = 'http://connect.garmin.com/proxy/activity-search-service-1.0/json/activities?' url_gc_gpx_activity = 'http://connect.garmin.com/proxy/activity-service-1.1/gpx/activity/' -url_gc_tcx_activity = 'http://connect.garmin.com/proxy/activity-service-1.1/tcx/activity/' +url_gc_tcx_activity = 'http://connect.garmin.com/proxy/activity-service-1.2/tcx/activity/' url_gc_original_activity = 'http://connect.garmin.com/proxy/download-service/files/activity/'

thedbexp avatar Aug 05 '16 20:08 thedbexp

@thedbexp That's not resolving for other filetypes: for gpx "endpoint is retired".

tymmej avatar Aug 06 '16 05:08 tymmej

GPX download was not working for me, changing the url_gc_gpx_activity to url_gc_gpx_activity = 'https://connect.garmin.com/modern/proxy/download-service/export/gpx/activity/'

solved the issue for me

jsanpedro avatar Aug 18 '16 19:08 jsanpedro

I'm getting error on this url which gets activities

http://connect.garmin.com/proxy/activity-search-service-1.0/json/activities

guys, does it work for you ?

frechina avatar Aug 31 '16 14:08 frechina

I think Garmin just shut down this endpoint. It could have been because of me. I just applied for access t to the Wellness API through the Garmin developer program (http://developer.garmin.com/wellness-api/overview) and in my application I mentioned my app and the informal API I'm using. Then a few days later it no longer works.

The reply I got from them, by the way, was

Thank you for your interest in the Garmin Developer Program. A publically available data privacy policy is a requirement to enter into the Developer Program. Once you have a publicly available data privacy policy, and are actively engaging end-users, please feel free to reapply at that time. Thanks!

ebrensi avatar Aug 31 '16 15:08 ebrensi

I don't believe the endpoint is down. When in a browser and logged into Garmin Connect I can load https://connect.garmin.com/proxy/activity-search-service-1.2/json/activities?start=0&limit=1 successfully. I think they must have changed the required cookie or auth headers in a way that is breaking the code. I haven't worked it out yet.

moderation avatar Aug 31 '16 15:08 moderation

@moderation you're right. However I checked all the login process and seems so similar than before

frechina avatar Aug 31 '16 17:08 frechina

I got it. You need to call these endpoint at the end of the login process

GET 'http://connect.garmin.com/modern'; GET 'https://connect.garmin.com/legacy/session';

frechina avatar Aug 31 '16 18:08 frechina

@frechina That worked! Amazing. How did you work that out? My heavily modified but working fork is at https://github.com/moderation/garmin-connect-export if anyone is interested.

moderation avatar Aug 31 '16 20:08 moderation

@moderation I used interceptor (from Postman) to capture all request made during Garmin login process. Then, after inspecting such amount of request I realised that the important ones were /modern and /legacy/session ;)

frechina avatar Sep 01 '16 05:09 frechina

@moderation Awesome https://github.com/moderation/garmin-connect-export working out of the box. I just needed to replace the old one with yours. Merci for the fork

johannesh83 avatar Sep 03 '16 07:09 johannesh83

thx a lot!

TobiasGirschick avatar Sep 08 '16 06:09 TobiasGirschick

Thanks everyone for finding the solution to the issues and implementing the fixes. I created a pull request containing the authentication and activity download fixes (no refactoring). I'm aware there were already forks working but I wanted to see if this could be merged into @kjkjava upstream repo.

Meanwhile you can use clone my fork https://github.com/fjgal/garmin-connect-export

fjgal avatar Sep 12 '16 01:09 fjgal

the service to get details of activity doesn't work any more (The requested endpoint is retired): https://connect.garmin.com/proxy/activity-service-1.3/json/activity/XXXXXXXXXXX Is there any new version of it?

elena321 avatar Sep 29 '16 16:09 elena321

Could it be this? https://connect.garmin.com/proxy/activity-service/activity/XXXXXXXX/

emtoonst avatar Dec 14 '16 08:12 emtoonst