serverless icon indicating copy to clipboard operation
serverless copied to clipboard

Support local postgres, consider `pg-cloudflare`?

Open ben-xD opened this issue 1 year ago • 8 comments

https://www.npmjs.com/package/pg-cloudflare was released 11 days ago. It uses native TCP connections on Cloudflare Workers, since Cloudflare has recently added that capability. https://www.npmjs.com/package/pg

However, this package uses pg, which doesn't take advantage of this.

Adding support for this will mean I can develop offline (in trains, flights) and improves the testability (e.g. just spin a postgres up to run certain tests).

ben-xD avatar May 26 '23 07:05 ben-xD

For now, you can run our WebSocket proxy locally (although this does mean changing some settings on the driver). Gal has blogged about this here: https://gal.hagever.com/posts/running-vercel-postgres-locally

It's on my list to see if we can support Cloudflare TCP in the driver too, though.

jawj avatar May 26 '23 10:05 jawj

Cool, I've got that working, thanks!

ben-xD avatar May 26 '23 18:05 ben-xD

Actually I'll re-open because although things work locally, adding support for Cloudflare connect() might be useful/interesting for Neon users.

ben-xD avatar May 26 '23 19:05 ben-xD

Sure. Of course, you should also be able to just use pg, but the SCRAM auth may burn CPU time.

jawj avatar May 28 '23 18:05 jawj

Is there a way to make the neon function using https fetch available with a local PG instance? I tried the pg proxy mentioned in the post above but I'm guessing this only supports WebSocket Pool right?

TimoWilhelm avatar Aug 22 '23 19:08 TimoWilhelm

@TimoWilhelm Take a look here for instructions on running our proxy locally: https://github.com/neondatabase/serverless/issues/33#issuecomment-1634853042

We do have plans to make this easier (e.g. with a Docker image or via the Neon CLI) in the future. And sooner than that, we have plans to allow embedding a http/WebSocket port in the query string part of the connection URL, so that no additional code is needed.

jawj avatar Aug 23 '23 12:08 jawj

Just ran into this as well. Until you guys find a better solution would it be possible to improve the error message, pointing you to the documentation about the proxy?

lukahartwig avatar Oct 30 '23 21:10 lukahartwig

I would consider this a very high impact improvement for developer experience. It is causing us a lot of friction that our local and test environments no longer work and that we have to ask all of our engineers to resort to frustrating workarounds.

neondb is ALMOST a drop-in replacement... once localhost just works, adopting neondb will become more of a no-brainer for startups

pg-cloudflare does seem promising so you don't have to force users to install a lot of extra things like Docker

venkatd avatar Nov 14 '23 18:11 venkatd

For now, you can run our WebSocket proxy locally (although this does mean changing some settings on the driver). Gal has blogged about this here: https://gal.hagever.com/posts/running-vercel-postgres-locally

It's on my list to see if we can support Cloudflare TCP in the driver too, though.

This no longer works in the current version ~0.10* but only does at ~0.9.*~. Even Vercel official Postgres local guide is outdated

mmikhan avatar Oct 14 '24 08:10 mmikhan