prisma-dart
prisma-dart copied to clipboard
medz
Prisma engine architecture mismatch
problem :
when generating prisma client on mac os or windows and deploying to linux machine the result engine works on the native machine , but when deployed it gives the ambiguous error : Exception: Prisma binary query engine not ready .
inspection :
after trying a lot of things to try to identify the problem I inspected that the problem is with file architecture , so I tired two solutions :
First try :
I tried to compile linux architecture in macOS locally so it will be deployed automatically with my dart_frog api docker image to google cloud run , i used :
generator client {
provider = "dart run orm"
binaryTargets = ["native" , "linux-musl"]
}
and it downloaded the linux binaries and generated the engine but didn't work when deployed 🤷🏻♂️
second try :
I tried to modify the docker file to install npx and generate the prisma engine on the deployed docker image itself as follows
# install node and npm
RUN apt-get update && apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_21.x | bash -
RUN apt-get install -y nodejs
# install npm packages
RUN npm install
RUN npx prisma generate
but it results in the following error :
> [build 12/12] RUN npx prisma generate:
0.465 node:fs:581
0.465 return binding.open(
0.465 ^
0.465
0.465 Error: ENOENT: no such file or directory, open '/app/node_modules/.bin/prisma_schema_build_bg.wasm'
0.465 at Object.openSync (node:fs:581:18)
0.465 at Object.readFileSync (node:fs:460:35)
0.465 at /app/node_modules/.bin/prisma:16:12782
0.465 at /app/node_modules/.bin/prisma:2:355
0.465 at Object.<anonymous> (/app/node_modules/.bin/prisma:242:122)
0.465 at Module._compile (node:internal/modules/cjs/loader:1378:14)
0.465 at Module._extensions..js (node:internal/modules/cjs/loader:1437:10)
0.465 at Module.load (node:internal/modules/cjs/loader:1212:32)
0.465 at Module._load (node:internal/modules/cjs/loader:1028:12)
0.465 at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12) {
0.465 errno: -2,
0.465 code: 'ENOENT',
0.465 syscall: 'open',
0.465 path: '/app/node_modules/.bin/prisma_schema_build_bg.wasm'
0.465 }
0.465
0.465 Node.js v21.6.2
------
Dockerfile:29
--------------------
27 | # install npm packages
28 | RUN npm install
29 | >>> RUN npx prisma generate
30 |
31 | # Build minimal serving image from AOT-compiled `/server` and required system
--------------------
ERROR: failed to solve: process "/bin/sh -c npx prisma generate" did not complete successfully: exit code: 1
@egyleader I understand your feelings very well. I am temporarily unable to solve the unpleasantness caused by this step. This requires Prisma to officially launch a new engine (currently being experimentally tried internally) that has not yet been announced to the public. Prisma currently has two engines: binary and NAPI. Obviously, the NAPI engine was developed for Node, and other languages can only use binary engines. Therefore, developers have to be required to download the corresponding engine on the appropriate architecture platform.
ok I understand the limitations of the current engine and the potential of the next one , but can we update the documentaion with warnings and current workarounds ??? the ability to use prisma with cloud providers is essential to any dart built api using prisma and will maximize the community of prisma-dart client and can push the development of this features and package further .
@egyleader Stability with cloud providers is difficult for me, my country is banned from using Google cloud and AWS! I personally don't have the funds to purchase and test it.
Often in this case I need help with ingestion, regarding compilation in Docker, maybe this file 👉 can help you https://github.com/NeroSong/dart_orm_docker_example/blob/master/Dockerfile
This file is an example written by a Prisma Client Dart user in the community when writing a blog .
Maybe you can refer to it, or maybe @NeroSong can help.
no , used the refrenced docker file and tried to get it to work but faced the same error
there is a problem in generating the prisma engine binary on server side RUN npx prisma generate
@egyleader I think because we need check connect with database DATABASE_URL="postgresql://postgress:xxxx@host:5432/test?schema=public" I can connect it when run local docker, but can't connect when deploy to vps cc @medz
I'll follow up, it seems that Prisma recently added architecture support, I don't know if this is the cause.
Maybe it won't be anytime soon, my wife just had a baby and I have my hands full.
FROM dart:stable AS build
WORKDIR /app
COPY pubspec.* ./
RUN dart pub get
COPY . .
RUN dart pub get --offline
# Install Node.js LTS to build prisma
RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - &&\
apt-get install -y nodejs
RUN npm install prisma
RUN npx prisma generate
RUN dart compile exe bin/dart_orm_docker_test.dart -o bin/server
# Copy Prisma Engine deps so to `/runtime/`
RUN FILES="libz.so libgcc_s.so libssl.so libcrypto.so"; \
for file in $FILES; do \
so="$(find / -name "${file}*" -print -quit)"; \
dir="$(dirname "$so")"; \
mkdir -p "/runtime${dir}"; \
cp "$so" "/runtime$so"; \
echo "Copied $so to /runtime${so}"; \
done
FROM scratch
COPY --from=build /runtime /
COPY --from=build /app/bin/server /app/bin/
COPY --from=build /app/prisma-query-engine /app/prisma-query-engine
WORKDIR /app
CMD ["./bin/server"]
FROM dart:stable AS build WORKDIR /app COPY pubspec.* ./ RUN dart pub get COPY . . RUN dart pub get --offline # Install Node.js LTS to build prisma RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - &&\ apt-get install -y nodejs RUN npm install prisma RUN npx prisma generate RUN dart compile exe bin/dart_orm_docker_test.dart -o bin/server # Copy Prisma Engine deps so to `/runtime/` RUN FILES="libz.so libgcc_s.so libssl.so libcrypto.so"; \ for file in $FILES; do \ so="$(find / -name "${file}*" -print -quit)"; \ dir="$(dirname "$so")"; \ mkdir -p "/runtime${dir}"; \ cp "$so" "/runtime$so"; \ echo "Copied $so to /runtime${so}"; \ done FROM scratch COPY --from=build /runtime / COPY --from=build /app/bin/server /app/bin/ COPY --from=build /app/prisma-query-engine /app/prisma-query-engine WORKDIR /app CMD ["./bin/server"]
this work for me with postgresql, thank you
@egyleader cc: @medz
Hai, did you solve this problem? I ran into the same problem but solved it. If not resolved, my thoughts may be helpful.
precondition
- Dart 3.2.0
- orm package ^4.1.0
- Gooogle Cloud Run
- RDB Planet Scale
conclusion
The conclusion was that we lacked the necessary dependent modules to run prisma-query-engine. This can only be noticed if you output the error log yourself. The log output seems to have room for improvement. Looking forward to the future ;)
solution
Not everything is correct. Please modify it for your environment. In my case, libssl1.1 was missing. so I will install that.
schema.prisma
generator client {
provider = "dart run orm"
engineType = "binary"
binaryTargets = ["debian-openssl-1.1.x", "native"]
}
Dockerfile
FROM dart:3.2.0
# Install Node.js LTS to builder
RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - &&\
apt-get install -y nodejs
RUN apt-get update && \
apt-get install -y wget
# install libssl1.1
RUN wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.0g-2ubuntu4_amd64.deb || (echo "Failed to download libssl1.1" && exit 1) && \
dpkg -i libssl1.1_1.1.0g-2ubuntu4_amd64.deb && \
rm libssl1.1_1.1.0g-2ubuntu4_amd64.deb
RUN apt-get install -y musl musl-dev && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Configure runtime for prisma
RUN FILES="libz.so libgcc_s.so libssl.so libcrypto.so"; \
for file in $FILES; do \
so="$(find / -name "${file}*" -print -quit)"; \
dir="$(dirname "$so")"; \
mkdir -p "/runtime${dir}"; \
cp "$so" "/runtime$so"; \
echo "Copied $so to /runtime${so}"; \
done
COPY /runtime/ /app
RUN npm install prisma
COPY /database/cloud/prisma-query-engine /app/prisma-query-engine
CMD ["dart", "hoge/api/main.dart"]
Inspection
I noticed this after checking the error log in process.
void _listenProcess(Process process) {
final stream =
process.stdout.transform(utf8.decoder).transform(const LineSplitter());
stream.listen((message) {
if (message.isEmpty) return;
try {
final data = json.decode(message);
if (data
case {
'level': "INFO",
'target': 'query_engine::server',
'fields': {'ip': final String host, 'port': final String port}
}) {
_endpoint = Uri(scheme: 'http', host: host, port: int.parse(port));
}
} catch (e, s) {
logger.severe('Unable to parse message: $message', e, s);
rethrow;
}
});
final streamStderr =
process.stderr.transform(utf8.decoder).transform(const LineSplitter());
streamStderr.listen((message) {
print(message);
});
}
and execute the ldd command and output the following log
ldd prisma-query-engine
linux-vdso.so.1 (0x00007ffc121e1000)
libssl.so.1.1 => not found
libcrypto.so.1.1 => not found
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f281d4ec000)
libc.musl-x86_64.so.1 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f281d30b000)
/lib/ld-musl-x86_64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x00007f281e6b3000)
can see that prisma-query-engine is missing some of the libraries it needs to run
I hope my comments here are helpful. If you have any questions, please let me know.
@yamarkz The binary engine is downloaded from Prisma CLI. Please remove the 'binaryTargets' setting. I did not find any remaining errors during Dockerfile testing.
If it still exists, can you give me the smallest reproduction repo?
Also, it would be best if you could upgrade to the latest version.
Due to the fact that I am the only one maintaining this library and my energy is limited, there is currently no idea of maintaining multiple versions.