cloud-sql-proxy icon indicating copy to clipboard operation
cloud-sql-proxy copied to clipboard

Add connectivity tests on startup to the actual instances

Open hilts-vaughan opened this issue 5 years ago • 10 comments
trafficstars

Some customers are confused by "Ready for new connections" and then try and connect and find it fails. This is often a routing problem but some of them incorrectly assume that the message above means everything was validated and fine.

Should we perform some kind of connectivity test when creating the instances or is this too much?

hilts-vaughan avatar Feb 12 '20 00:02 hilts-vaughan

I think we should definitely provide this feature. It probably should be toggle-able by a flag, and I'm unsure if it should be enabled by default. I'd be open to other folks opinions on the matter.

kurtisvg avatar Feb 12 '20 15:02 kurtisvg

I think unless all the examples show the flag it won't see widespread adoption and many users will still confuse themselves. I would be inclined for default but it may not technically be backwards compatible if for some reason on startup the proxy can't connect (but presumably it is later able to)

hilts-vaughan avatar Feb 12 '20 15:02 hilts-vaughan

Yes please! I'm experience an issue with this now. Randomly my container is failing with OAuth and finally got into a broken container today to look around. There's an issue with SSL certs. Yet will continue along saying "Ready for new connections" ... Which it's not. While I'd really love to know why sometimes it's unable to make any HTTPS requests, in the least a failure would help a lot. It could re-deploy a working container.

tmaiaroto avatar Feb 28 '20 17:02 tmaiaroto

Hey Kurtis,

In the case where we detect failed connectivity on instances, what should we do? What if there are multiple instances only one of them fail? Crash? Emit a warning?

hilts-vaughan avatar Feb 29 '20 01:02 hilts-vaughan

As a developer, I would find it most useful if it failed and emitted a reason why.

kurtisvg avatar Feb 29 '20 20:02 kurtisvg

Maybe we could implement this as just a basic TCP connectivity test, since that would let us remain agnostic of the actual underlying protocols?

This would rule out routing and firewall problems, but of course not catch EVERYTHING that could go wrong.

hilts-vaughan avatar Mar 01 '20 05:03 hilts-vaughan

Not being able to connect to the OAuth endpoint is a pretty big thing to go wrong though. Would love to see it fail should that happen. If I can get the time, I may take a look and see if I can add a pull request or something.

tmaiaroto avatar Mar 03 '20 01:03 tmaiaroto

I think you're right -- we should try and catch that too. Though, let's fry one fish at a time. :)

hilts-vaughan avatar Mar 07 '20 00:03 hilts-vaughan

Related to https://github.com/GoogleCloudPlatform/cloudsql-proxy/issues/137.

enocom avatar Feb 09 '21 23:02 enocom

This would be an easy feature to add now that we have a CheckConnections method on the internal proxy struct. We currently use this when health checks are enabled. It would be easy to allow callers to opt-in to a connection check on proxy startup.

enocom avatar Aug 29 '22 20:08 enocom