prisma
prisma copied to clipboard
prisma
Dynamic Connection Settings
Problem
When connecting to AWS RDS using IAM authentication, a token is generated that is used as the password for new connections. This token expires after 15 minutes. After that, the token can no longer be used to create new connections.
Existing connections are still valid.
With Prisma, it seems like connection settings including the password must be static, and can not be altered after the PrismaClient has been created.
Suggested solution
With Knex, the connection settings do not need to be static. An async function can also be passed in that will provide the connection settings when called. This can include an expiration checker that is called when a new connection is created. If the settings are expired, the connection settings provider can be called again to update the connection settings.
This should not close existing connections in the pool. It should only affect the settings of new connections that are created.
It might look like this in Prisma.
export prisma = new PrismaClient({dataSources: { db: generateConnectionSettings }})
async function generateConnectionSettings(){
const token = await getIAMToken(auth)
const expiresAt = Date.now() + 840000 // Expires in 14 minutes
return {
url: <connection string with generated token>,
expirationChecker: () => {
return expiresAt <= Date.now()
}
}
}
Alternatives
We are currently creating a new PrismaClient for every http request, but this is only possible because we have an internal only tool with very low request volume. This is very wasteful.
Additional context
https://github.com/prisma/prisma/discussions/4147
Just to figure out the complexity here: Would it also be viable to use a function prisma.updateConnectionString() that overwrites the currently used connection string somehow inside the Prisma Client to be used for new connection?
I think that would work in my situation. However, this might create extra work in a case where creating the new connection string was costly and the refresh interval is low and connections are not recreated often.
That seems like a rare situation, however.
It is definitely better than creating a new Prisma client for every request to our webserver, which is what we currently do.
@kevinchevalier Have you considered reacting to requests failing with a permission error (of course depends on what this really looks like) and only then recreate Prisma Client? I think that would probably be doable in a middleware, and also getting and setting the new token could be handled there until this is implemented.
why not passing in a password with:
password: string | () => Awaited<string>
then prisma could just generate the password whenever a new connection is to be opened:
const connectionPassword = (typeof password === "string") ? password : await password();
Because connections are not opened in Javascript context, but the underlying Query Engine. Having to go back to JS every time a connection is opened would add considerable communication effort between the components which is not easy to implement in the current way things are set up unfortunately.
OK, understood.
Then I also like the idea of providing something like prisma.updateConnectionString() or prisma.updatePassword(new_pw: string), which should allow pushing new passwords into the query engine... but yeah, there is this risk for creating the new tokens too often in unnecessary cases.
Could the query engine first try to use the currently set password, and only if the connection attempt fails (with invalid login or so) calling back into JS for retrieving a new password (if the password is provided as a function within the JS context in the first place)?
The "call back" part is where the current architecture would not be helpful unfortunately.
But similar idea is what I suggested here: https://github.com/prisma/prisma/issues/7869#issuecomment-964429877 It would just mean that the Client side itself reacts to the error, gets a new token, and then just tries again.
It would be great to have this feature which seems to be already implemented in other Node.js ORMs. We currently have the same issues working with Prisma and PostgreSQL on Azure with @azure/identity.
Do we have any idea when this might be supported? About to have to rip and replace Prisma for TypeORM or something (which I really don't want to do) because we can't find a doable way around this.
Just wanted to add another issue that the current workaround to use AWS IAM has brought up: Not being able to use the default Sentry tracing integration.
That integration takes a Prisma client instance once on initialization, but since we need to create a new instance every 15 minutes, tracing will stop working then at the latest (but in practice immediately since we have to create a dummy instance immediately to get the dmmf).
Any updates on this? Is there a way to accomplish this without having to create a new PrismaClient each time the token expires?
@janpio
It would just mean that the Client side itself reacts to the error, gets a new token, and then just tries again.
would you have time to write quick example of what that might look like using extensions? I'm not sure how to replace the current client instance and re-query using extensions.
I would assume something like (fully untested):
... // somehow react to connection error via try/catch etc
prisma.$disconnect()
prisma = null
prisma = new PrismaClient({
datasources: {
db: {
url: ...',
},
}
})
...
Uses https://www.prisma.io/docs/reference/api-reference/prisma-client-reference#programmatically-override-a-datasource-url to create a new Client with a specific connection string. That should close down the old one, and then overwrite it with a new one.
Any news on when this feature will be actively worked on? Will be very helpful for establishing connections to AWS RDS or Cloud SQL that can automatically refresh OAuth2 tokens for IAM authentication 🙏
We would like this feature as well it's very important to the work we're doing.
@kevinchevalier as a workaround could you not create a singleton class for managing the connection and create a new client only when the expiry is about to take place?
@kevinchevalier as a workaround could you not create a singleton class for managing the connection and create a new client only when the expiry is about to take place?
That is a possible workaround, but it is slightly less efficient. The reason is that while the password changes every 15 minutes, open connections are still valid forever, until they are closed. If we destroyed the client every 15 minutes, it wouldn't be as efficient.
I understand that this inefficiency is unlikely to have a large impact. Heck, we create a new client for every request and it is fine for our application. But it would be great to have this functionality built in to regenerate connection settings.
Coming back to this issue after 3 years, one of the many reasons I never adopted Prisma, it's sad to see we are still talking about workarounds 🥲
Needlessly recreating connections and/or swapping out prisma instances at runtime is wasteful, adds complexity and is error prone. 🤞 this gets solved properly in 2024 🤞
Would really need a solution for our project as well. We need to connect to a PostgreSQL database with an IAM user and token that expires after 15min. For now, we are connecting to the database with a username and password that is setup as secrets in AWS with secret rotation to increase security and we need to trigger the application to restart when the secret rotates. It would be much better and more secure to be able to connect with IAM auth.
Would really need a solution for our project as well. We need to connect to a PostgreSQL database with an IAM user and token that expires after 15min. For now, we are connecting to the database with a username and password that is setup as secrets in AWS with secret rotation to increase security and we need to trigger the application to restart when the secret rotates. It would be much better and more secure to be able to connect with IAM auth.
In need of this NOW as well +1
We're using prisma with IAM and lambda and encountering this error. We're also waiting just before the IAM timeout of fifteen minutes, and then refetching the token and creating a new prisma client, but it's not great.
