mongodb-client-encryption in CSFLE

[thismehmood]

> import { Binary, ClientEncryption, MongoClient, ObjectId } from 'mongodb';
> import { encryptData, decryptData } from './gcpKmsClient';
> import * as crypto from 'crypto';
> 
> const uri = process.env.MONGODB_URI;
> const client = new MongoClient(uri);
> 
> export class KeyVaultService {
>   private keyVaultNamespace = 'encryption.keyVault';
> 
>   constructor() {
>     client.connect();
>   }
> 
>   async createDataEncryptionKey(): Promise<ObjectId> {
>     const keyVault = client.db('encryption').collection(this.keyVaultNamespace);
> 
>     const dek = crypto.randomBytes(32);
>     const encryptedDek = await encryptData(process.env.GCP_KEY_NAME, dek);
> 
>     const keyDocument = {
>       _id: new ObjectId(),
>       keyMaterial: encryptedDek,
>       creationDate: new Date(),
>     };
> 
>     await keyVault.insertOne(keyDocument);
>     return keyDocument._id;
>   }
> 
>   async getDataEncryptionKey(keyId: string): Promise<Buffer> {
>     const keyVault = client.db('encryption').collection(this.keyVaultNamespace);
>     const keyDocument = await keyVault.findOne({ _id: new ObjectId(keyId) });
>     if (!keyDocument) {
>       throw new Error('Key not found');
>     }
>     const dek = await decryptData(process.env.GCP_KEY_NAME, keyDocument.keyMaterial);
>     return Buffer.from(dek, 'base64');
>   }
> 
>   async encryptField(dek: Buffer, field: any): Promise<Binary> {
>     const encryption = new ClientEncryption(client, {
>       keyVaultNamespace: this.keyVaultNamespace,
>       kmsProviders: {
>         gcp: {
>           email: process.env.GCP_SERVICE_ACCOUNT_EMAIL,
>           privateKey: process.env.GCP_SERVICE_ACCOUNT_PRIVATE_KEY,
>         },
>       },
>     });
> 
>     const encryptedField = await encryption.encrypt(field, {
>       keyId: new Binary(dek, Binary.SUBTYPE_BYTE_ARRAY),
>       algorithm: 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic',
>     });
> 
>     return encryptedField;
>   }
> }

here is my code I'm trying to CSFLE using GCP kms, and we have microservices runing on docker environment so "mongodb-client-encryption": "^6.1.1", is already present in package.json and node_modules as well. but when i run the docker environment i still got the issue on my terminal.

> patient-1       | [11:52:12.204] ERROR (50): Optional module `mongodb-client-encryption` not found. Please install it to use auto encryption or ClientEncryption. {"context":"RpcExceptionsHandler"}
> patient-1       |     err: {
> patient-1       |       "type": "Error",
> patient-1       |       "message": "Optional module `mongodb-client-encryption` not found. Please install it to use auto encryption or ClientEncryption.",
> patient-1       |       "stack":
> patient-1       |           MongoMissingDependencyError: Optional module `mongodb-client-encryption` not found. Please install it to use auto encryption or ClientEncryption.
> patient-1       |               at getMongoDBClientEncryption (/usr/src/app/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/mongodb/src/deps.ts:279:26)
> patient-1       |               at Function.getMongoCrypt (/usr/src/app/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/mongodb/src/client-side-encryption/client_encryption.ts:86:50)
> patient-1       |               at ClientEncryption (/usr/src/app/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/mongodb/src/client-side-encryption/client_encryption.ts:143:41)
> patient-1       |               at KeyVaultService.encryptField (/usr/src/app/dist/apps/patient/main.js:5904:28)
> patient-1       |               at PatientService.addMedicalHistory (/usr/src/app/dist/apps/patient/main.js:1532:48)
> patient-1       |               at async /usr/src/app/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_class-transformer@0_wf7kzsnxi4ek5cejwuautyv53q/node_modules/@nestjs/microservices/context/rpc-proxy.js:11:32
> patient-1       |               at async ServerTCP.handleMessage (/usr/src/app/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_class-transformer@0_wf7kzsnxi4ek5cejwuautyv53q/node_modules/@nestjs/microservices/server/server-tcp.js:67:54)
> patient-1       |     } 

Originally posted by @thismehmood in #9883

[vkarpov15]

It doesn't look like you're using Mongoose?

[thismehmood]

It doesn't look like you're using Mongoose?

Actually, I am using Mongoose. The code above defines a KeyVaultService where I pass the required field to be encrypted. I have also created a key vault in MongoDB, where the Data Encryption Key (DEK) for each patient is stored.

I am running the environment locally, and it works fine. However, inside Docker, I am facing the following issue. Do you have any insights on how to resolve it? Why is mongodb-client-encryption not being detected in the Docker environment even though it is installed? Are there any known compatibility issues between mongodb-client-encryption and the Docker setup? Could this be related to missing native dependencies or an issue with pnpm inside the container? Any insights or suggestions would be greatly appreciated.

[thismehmood]

@vkarpov15 @wvl @svperfecta

[vkarpov15]

Which version of Mongoose? Can you also please run npm list mongoose and paste the output? Also, can you share your Dockerfile please?

[thismehmood]

Base image for development

FROM node:18.20-slim AS development

Install necessary dependencies for Puppeteer and MongoDB Client Encryption

RUN apt-get update && apt-get install -y
chromium
ca-certificates
libc6
libx11-xcb1
libxcb1
libxcomposite1
libxcursor1
libxdamage1
libxi6
libxtst6
libnss3
libxrandr2
libgbm1
libasound2
libatk1.0-0
libatk-bridge2.0-0
libdrm2
libxshmfence1
libnspr4
fonts-liberation
libappindicator3-1
xdg-utils
python3
build-essential
openssl
pkg-config
&& apt-get clean && rm -rf /var/lib/apt/lists/*

Set the working directory

WORKDIR /usr/src/app

Copy relevant files

COPY package.json pnpm-lock.yaml ./ COPY tsconfig.json nest-cli.json ./

Install pnpm globally

RUN npm i -g pnpm

Install all dependencies (including optional native dependencies)

RUN pnpm install --no-frozen-lockfile

Approve builds (should happen after install)

RUN pnpm approve-builds

Rebuild specific dependencies (mongodb-client-encryption)

RUN pnpm rebuild mongodb-client-encryption

Optional: rebuild everything if needed (less recommended)

RUN pnpm rebuild

Copy app and libs

COPY apps/patient apps/patient COPY libs libs

Build the patient application

RUN pnpm build patient

[thismehmood]

here is my docker file and mongodb-client-encryption is optional module while installing dependencies pnpm i so in this docker file, I used this RUN pnpm approve-builds. but right now while running in the docker I'm getting the error MongoCryptError: KMS request closed,

[vkarpov15]

Glad pnpm approve-builds fixed your issue. mongodb-client-encryption is an optional peer dependency of mongodb, so you may need some extra work to install it like described here: https://stackoverflow.com/questions/72468635/pnpm-peer-dependencies-auto-install.

Re: the KMS request closed error, I will need a more complete repro script. I'll at least need the code from gcpKmsClient and a complete dockerfile

[baileympearson]

The KMS request closed error might be the same error fixed in driver 6.14.2: https://github.com/mongodb/node-mongodb-native/releases/tag/v6.14.2

[github-actions[bot]]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue was closed because it has been inactive for 19 days and has been marked as stale.