prisma
prisma copied to clipboard
prisma
Support for AWS Secrets Manager or Azure KeyVault in `schema.prisma`
Problem
I'm using Prisma only to manage my database schema, not for the CRUD operations in code. The problem is that the schema.prisma file only supports defining the datasource URL using values defined in a file (dotenv). Since I don't want to store the database credentials in my project I'd like to obtain them from AWS Secrets Manager service.
Suggested solution
Provide a mechanism like a callback function to defined the datasource URL.
That is very reasonable. Do you have an implementation of this in mind or examples how other tools solve this?
That is very reasonable. Do you have an implementation of this in mind or examples how other tools solve this?
I'd imagine something like this:
datasource db {
provider = "postgresql"
url = await getDatabaseUrl();
}
function getDatabaseUrl() {
// obtain credentials from cloud
// create database url based on obtained credentials
// return databaseUrl;
}
Ok, so being able to call out to a getDatabaseUrl.js or .ts that executes the logic to return the credential would work for you?
I used SM for a client, you basically need a small script to inject environment variables before calling the prisma command. Unsure if it is really the job of prisma to do that? but the hook to a ts/js file seems resonable.
Can you share the script and how you called it as a current workaround? That could help @jquirossoto a bit - as we will not implement this tomorrow obviously but more long term.
I dont have access to it, but here is an exemple using GCP SM that is similar:
const fs = require('fs');
const path = require('path');
const util = require('util');
const { GoogleAuth } = require('google-auth-library');
const { SecretManagerServiceClient } = require('@google-cloud/secret-manager');
const writeFile = util.promisify(fs.writeFile);
const mkdir = util.promisify(fs.mkdir);
const auth = new GoogleAuth();
const client = new SecretManagerServiceClient();
const SECRETS = {
DATABASE_URL: (env) => `myproject-${env}-database-url`,
};
const ENVIRONMENTS = {
production: 'prod',
staging: 'staging',
};
const getSecretValue = async (name) => {
const projectId = await auth.getProjectId();
const secretVersionName = `projects/${projectId}/secrets/${name}/versions/latest`;
const [secret] = await client.accessSecretVersion({
name: secretVersionName,
});
const payload = secret.payload.data.toString('utf8');
return payload;
};
async function main() {
const env = ENVIRONMENTS[process.env.NODE_ENV];
if (!env) {
throw new Error('Unsupported environment');
}
// Fetch secrets
const lines = [];
for (const [localName, gcpName] of Object.entries(SECRETS)) {
const secretName = gcpName(env);
const secretValue = await getSecretValue(secretName);
lines.push(`${localName}=${secretValue}`);
}
// Write file
const content = lines.join('\n');
const envPath = path.join(__dirname, '..', 'config', 'production.env');
await mkdir(path.dirname(envPath), { recursive: true });
await writeFile(envPath, content);
}
main().catch((err) => {
console.log(err);
process.exit(1);
});
This works well for containers were the production.env is discarded when it dies.
For your need I would do something like:
DATABASE_URL=$(gcloud secrets versions access latest --secret some-secret) prisma migrate
DATABASE_URL=$(aws secretsmanager get-secret-value --secret-id some-secret --query SecretString --output text) prisma migrate
You can even wrap that in a package.json script.
@janpio was there any progress or further insight on this issue? We are looking at using the secrets manager similarly and just wanted to check that @Sytten's approach is still the workaround one would be using.
UPDATE: due to using docker images, the above workaround did not work in our use case.
No update or it would be mentioned in this issue (or the issue would optimally be closed).
+1. blocked on the same issue. need to be able to use this with docker as well as keeping secrets in the secrets manager. pretty frustrated.
We ended up with a hacky solution for docker and Prisma. Our Docker file copies over a shell script: db_wrapper.sh that we then execute before running our nextjs app: CMD ["/app/db_wrapper.sh", "npm", "start"]
#!/usr/bin/env sh
export DATABASE_URL="postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE"
exec "$@"
We are using the aws cdk to pass in the secrets:
...
new PEcsService(appStack, "PEcsService", {
serviceName: "PEcsService",
healthCheckPath: "/",
replicaCount: 1,
subdomain: "ecs",
container: {
containerName: "EcsContainer",
image: ecs.ContainerImage.fromAsset(path.join(__dirname, "..", "..")),
portMappings: [{ containerPort: 3000 }],
environmentFiles: [],
environment: {
PGDATABASE: "ecs",
},
secrets: {
PGHOST: ecs.Secret.fromSecretsManager(db.secret, "host"),
PGPORT: ecs.Secret.fromSecretsManager(db.secret, "port"),
PGUSER: ecs.Secret.fromSecretsManager(db.secret, "username"),
PGPASSWORD: ecs.Secret.fromSecretsManager(db.secret, "password"),
},
},
})
...
Thank you I was just reaching the same solution Migration is fine, but I'm looking at the cause because an error occurs in prisma studio.
This feature would be awesome and help us with setting up local environments as well as CI/CD. Also this would help us creating the Prisma client in a serverless GraphQL environment as well.
Meanwhile, we created a bash script to help us running migrations and studio locally.
#!/usr/bin/env sh
# set-db-url.sh
SECRET_ARN="..."
SECRET_BLOB=$(aws secretsmanager get-secret-value --output=text --query SecretString --secret-id "$SECRET_ARN")
PGUSER=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).username' <<< "$SECRET_BLOB")
PGPASSWORD=$(node -pe 'encodeURIComponent(JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).password)' <<< "$SECRET_BLOB")
PGHOST=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).host' <<< "$SECRET_BLOB")
PGPORT=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).port' <<< "$SECRET_BLOB")
PGDATABASE=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).dbname' <<< "$SECRET_BLOB")
export DATABASE_URL="postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE"
exec "$@"
And then using it with our npm scripts like so:
"prisma:studio": "scripts/set-db-url.sh prisma studio"
any update on if a solution for this might be forthcoming? We're deploying to GKE (so also using Docker), and using GSM to manage secrets, also not wanting to keep our db credentials in env variables.
This is very necessary as well for my company's use case. We're doing some wacky things to get this to work.
Any update on this?
Im tried calling the Prisma client in my handler using my AWS Secret Manager function, but i am getting a schema.prisma file not found when separating the schema.prisma and the client
An async callback to fetch the database url would be awesome for Prisma running on serverless functions like AWS Lambda. clients like Knex have this as well https://knexjs.org/guide/#configuration-options
+1 It would be great if we could have something like FILE() just like ENV() function. or maybe an approach to create a custom function as well and use it anywhere... (something like an extension or plugin to add custom functionality to the schema)
We had to drop using Prisma for all of our services because it lacks this functionality....
I'm about to drop Prisma because of this too.
I think you can make a workaround, by creating a bash script to read the secret, set it as env, and execute the prisma cli command. just like what this person did in this comment (in the current issue): https://github.com/prisma/prisma/issues/7534#issuecomment-1047745694
or if you can also get help from ChatGPT :)
We are currently looking into the design of a library that gives access to Prisma Migrate from code without depending on the CLI. If this sounds relevant to you, please help us with this 5 min survey so we can learn more about your specific needs.
(Note: the current scope is Prisma Migrate and it would not help when you need to feed a URL into Prisma Client)
http://pris.ly/orm-survey/migrate-sdk
🙇 Thank you very much!
