oauth2: Listen on random port for redirect_uri http://localhost

[JohannesFKnauf]

The happy.oauth2-capture-redirect/wait-for-redirect port selection logic expects one or multiple complete redirect URL incl. port number.

In my CLI time-tracking tool parti-time I need to connect to Google Sheets API. In Google's "APIs and Services" configurations I configured a Client ID for a Desktop app, just as described in the Google Sheets API Java Client quickstart guide. The Google Sheets API Java Client's design isn't a good fit for clojure idiomatic design. Also, it's breaking my graalvm native image build -- probably because of its extensive use of reflection. I found happygapi and I'm switching right now.

When downloading the client secret JSON, the redirect-uris are configured as http://localhost -- without any port. happygapi defaults to port 80. This is particularly bad, because port 80 is a privileged port that requires root permissions.

My current workaround when using happygapi: I add an artificial fixed port value on the redirect-uri before passing it to happy.oauth2-credentials/init!.

When running a desktop or CLI application, the server on localhost will be ephemeral. Also, you won't necessarily know which ports are free on your PC. From my perspective, the natural port choice in this scenario is a random port.

Rough implementation sketch

ring.adapter.jetty/run-jetty returns us an org.eclipse.jetty.server.Server. The port for the HTTP connector (a.k.a. ServerConnector) is taken directly from the options. According to the docs,

Port this connector listens on. If set the 0 a random port is assigned which may be obtained with getLocalPort()

In other words, we should be able to pass the magic port number 0 and get the actual local port from the Jetty server object resp. from its connectors. We need that port, to pass it as port of the redirect URL in the OAuth request.

Remarks

If you can just confirm that I didn't overlook anything and my idea is valid, but you don't have the time to implement the change, I'd be happy to file a PR.

[timothypratley]

Hi @JohannesFKnauf

Thank you for bringing this to my attention.

The redirect_uri configured via the Cloud Console implies port 80. i.e: http://localhost/redirect implies http://localhost:80/redirect

Using another port is possible by explicitly configuring a different port: http://localhost:8080/redirect

http://localhost/redirect is probably the default, but that it is easy to update the configuration via Cloud Console.

I believe that if you add the above configuration then download or update the secret.json then everything should work just fine because HappyGAPI will detect the port from the redirect_uri and listen on that port instead. (Happy GAPI uses the last uri if there are multiple). If this is a good solution for you, I would appreciate a PR to update the README to reflect that specifying a port is best. Alternatively I'd be happy to investigate further.

I suspect that Google will not authenticate with a random port, because the redirect_uri would not exactly match the configured value. Random would be better because it avoids the possibility of 8080 collision (or whatever number is configured), but I don't think it is possible. I'd love to be proven wrong about this though! It's certainly worth trying to confirm or deny whether the port is considered part of the identity of the uri. To check it, you should be able to edit the secret.json and add a random number that is not configured and see if it works. If it does work, I'd love to merge a PR working as you described.

My understanding of the uri is based on this documentation: https://developers.google.com/identity/protocols/oauth2/web-server

redirect_uri Required Determines where the API server redirects the user after the user completes the authorization flow. The value must exactly match one of the authorized redirect URIs for the OAuth 2.0 client, which you configured in your client's API Console Credentials page. If this value doesn't match an authorized redirect URI for the provided client_id you will get a redirect_uri_mismatch error. Note that the http or https scheme, case, and trailing slash ('/') must all match.

When configuring an oauth2 key it is possible to select an "Application type". I've been using the "Web application" type, but there is also a "Desktop application" type. Strangely it does not seem to be possible to configure the redirect_uri for "Desktop application" type credentials. Perhaps there is a detail here that is worth investigating. Perhaps the desktop key is more permissive in allowing other ports???

If you are distributing an app please bear in mind that it is probably best not to have a secret.json text file, but rather put the configuration in the code. While this won't completely protect your secret, it should make it more difficult to discover.

Sincerely, Timothy

[JohannesFKnauf]

Oh wow! I didn't expect such a quick response. Highly appreciated!

You are raising very valid points. I've indeed only been using the "Desktop Application" type, so far. I did some research. Let me summarize my findings and experiences with parti-time.

Why do random ports work for desktop applications?

From RFC 8242: OAuth 2.0 for Native Apps:

The authorization server MUST allow any port to be specified at the time of the request for loopback IP redirect URIs, to accommodate clients that obtain an available ephemeral port from the operating system at the time of the request.

From Google's OAuth 2.0 for Mobile & Desktop Apps documentation about the redirect_uri request parameter

Loopback IP address http://127.0.0.1:port or http://[::1]:port

Query your platform for the relevant loopback IP address and start an HTTP listener on a random available port. Substitute port with the actual port number your app listens on.

The alternative to using loopback URLs would be using a custom application scheme, backed by a call to some parti-time command. The problem with that approach is, that it requires the user to modify their OS configuration, i.e. register a new protocol handler.

I don't know, if that behaviour for Google API clients is limited to Desktop application clients or if it always applies for http://localhost redirect URIs e.g. also for web application clients.

How does parti-time distribute client credentials?

parti-time doesn't distribute client credentials -- neither through checked-in files nor embedded in the code. Instead, it requires users to setup their own OAuth Client in the developer console and provide the client secrets in ~/.config/parti-time/credentials.json.

How did parti-time use random ports with Google Client libs in the past?

In the previous version of parti-time's google sheets client module, I've been using the .setPort(-1) option in LocalServerReceiver.Builder. That worked fine (including the redirect), but the Google API Client libs are cumbersome to use and break graalvm native-image builds.

How does parti-time now override ports with happygapi?

The current version of parti-time's google sheets client module appends a static port to the http://localhost entry. The redirect there works fine as well.

The issue is, that I can't append a random port, because I don't know in advance which one is free. When running a web server like Jetty, that web server delegates opening a ephemeral random available port to the underlying platform libs, which delegate it further to the OS. To leverage that, it has to be done at server creation time, e.g. for Jetty by specifying the magic value 0 for the port number.

[timothypratley]

@JohannesFKnauf Ah I see, great.

Yes, I agree that your proposed implementation plan will work correctly and be much better than the current situation.

Thank you so much for researching that and volunteering to fix it. I'd be very grateful to receive a PR with the change.

I think the idea is changing happy.oauth2-capture-redirect/wait-for-redirect:

1. It should specify a port 0 to get a random port
2. It should extract the port from server result jetty/run-jetty
3. It request the code using the random port

Please don't hesitate to let me know if you get stuck; maybe I can help in some way.

[timothypratley]

Released as 0.4.10 https://clojars.org/happygapi/versions/0.4.10

[timothypratley]

@JohannesFKnauf I'm planning on releasing 2 new libraries that address some of the design mistakes I made;

https://github.com/timothypratley/happyapi (focuses on being an oauth2 client, middleware oriented) https://github.com/timothypratley/happygapi2 (uses the new happyapi client, better function signatures and docs)

And here's an example showing how I used it to get some YouTube data:

https://timothypratley.github.io/happyapi/happy.notebook.youtube_clojuretv.html

I'd like to get your thoughts on whether these changes make sense, and if they would benefit your use case?