azure-cosmos-db-emulator-docker
azure-cosmos-db-emulator-docker copied to clipboard
Azure
Linux container fails to start up with SIGABRT
Describe the bug The Linux container fails to start up with SIGABRT.
To Reproduce Steps to reproduce the behavior:
docker run \
--publish 8081:8081 \
--publish 10250-10255:10250-10255 \
--interactive \
--tty \
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
Expected behavior The container should start up without errors.
Screenshots
$ docker run \
--publish 8081:8081 \
--publish 10250-10255:10250-10255 \
--interactive \
--tty \
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
Unable to find image 'mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest' locally
latest: Pulling from cosmosdb/linux/azure-cosmos-emulator
d7bfe07ed847: Pull complete
520d4c735f8c: Pull complete
0722c4f37e97: Pull complete
f1a6e15ab569: Pull complete
6f9ffa3152bf: Pull complete
cc3063780407: Pull complete
14cd358b5760: Pull complete
dfaba22a8450: Pull complete
5a96b88a19ba: Pull complete
f4133c361e5f: Pull complete
85c85b80785c: Pull complete
74e2278670c0: Pull complete
3e169396e112: Pull complete
6291d3a77d8a: Pull complete
4f4fb700ef54: Pull complete
712692508376: Pull complete
1b4ec4f9b41f: Pull complete
86e8e6ee474c: Pull complete
75e82f426270: Pull complete
a26d85cd5ccf: Pull complete
Digest: sha256:85c7b1771de120d3923c077606383e52b3763a7fe740c9f13bb2a95b5cb841e9
Status: Downloaded newer image for mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
This is an evaluation version. There are [96] days left in the evaluation period.
Starting
This program has encountered a fatal error and cannot continue running at Tue Jan 23 10:13:34 2024
The following diagnostic information is available:
Reason: Fatal Signal (0x00000001)
Signal: SIGABRT - Aborted (6)
Stack:
SP IP Function
---------------- ---------------- ----------------
00007b28b1b76fa0 00005d8b4ef95bfa <unknown>
00007b28b1b78000 00005d8b4ef955cf <unknown>
00007b28b1b78260 00005d8b4ef94a61 <unknown>
00007b28b1b78280 00007b28b61eb090 killpg+0x40
00007b28b1b78e60 00007b28b61eb00b gsignal+0xcb
00007b28b1b78f80 00007b28b61ca859 abort+0x12b
00007b28b1b790b0 00005d8b4ef2eaa2 <unknown>
00007b28b1b79160 00005d8b4efaf657 <unknown>
Process: 19 - cosmosdb-emulator
Thread: 126 (application thread 0x1c8)
Instance Id: 5a3e9f7a-43ba-4a11-9c9e-55ad1e163626
Crash Id: 56eda8b5-e7d6-421f-8478-351b2a16042b
Build stamp: (null)
Distribution: Ubuntu 20.04.6 LTS
Processors: 16
Total Memory: 33028304896 bytes
Timestamp: Tue Jan 23 10:13:34 2024
Last errno: -34938881
Last errno text: Unknown error -34938881
*********** PAL PANIC CORE DUMP GENERATION FAILED **********
Unable to locate handle-crash.sh. Error: File: signals.cpp:483 [Status: 0xC0000034 Object name not found errno = 0x2(2) No such file or directory]
Executing: /usr/local/bin/cosmos/handle-crash.sh with parameters
handle-crash.sh
/usr/local/bin/cosmos/cosmosdb-emulator
19
/usr/local/bin/cosmos
/tmp/cosmos/appdata/log/
5a3e9f7a-43ba-4a11-9c9e-55ad1e163626
56eda8b5-e7d6-421f-8478-351b2a16042b
*********** PANIC CORE DUMP GENERATION FAILED **********
Attempt to launch handle-crash.sh failed.
This program has encountered a fatal error and cannot continue running at Tue Jan 23 10:13:34 2024
The following diagnostic information is available:
Reason: Host Extension RTL_ASSERT (0x00000003)
Status: STATUS_WAIT_2 (0x00000002)
Message: !killTheTarget
Stack:
SP IP Function
---------------- ---------------- ----------------
00007b28b3dfbdf0 00005d8b4ef95bfa <unknown>
00007b28b3dfce50 00005d8b4ef955cf <unknown>
00007b28b3dfd0b0 00005d8b4eeffb56 <unknown>
00007b28b3dfd0e0 00005d8b4ef990e4 <unknown>
00007b28b3dfe310 00005d8b4ef987a9 <unknown>
00007b28b3dfe440 00007b28b6846609 start_thread+0xd9
00007b28b3dfe500 00007b28b62c7133 clone+0x43
Process: 17 - cosmosdb-emulator
Thread: 18
Instance Id: 5a3e9f7a-43ba-4a11-9c9e-55ad1e163626
Crash Id: 56eda8b5-e7d6-421f-8478-351b2a16042b
Build stamp: (null)
Distribution: Ubuntu 20.04.6 LTS
Processors: 16
Total Memory: 33028304896 bytes
Timestamp: Tue Jan 23 10:13:34 2024
Last errno: 2
Last errno text: No such file or directory
Aborted (core dumped)
Desktop (please complete the following information):
- OS: Arch Linux
- Browser N/A
- Version N/A
Docker Images Used:
- Linux
Arguments && Environment variables to start Docker:
/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
Docker Environment
- Standalone
Additional context
This has been working sometime near the the end of 2023.
$ docker --version
Docker version 24.0.7, build afdd53b4e3
@szszoke can you please confirm the OS , whether it is ubuntu or arch linux?
The host OS running Docker itself is Arch Linux, the Cosmos DB image seems to be based on an Ubuntu image.
I have the same issue, also on Arch Linux.
I'd like to use an older version of the image in the meantime because at some point in the past it did work. What are some tags I can use besides latest? I can't seem to find a list of tags in the registry.
The latest image hasn't been updated since 2023 November 03 and I remember this working after that.
In the meantime, I found an issue on MS SQL that looked very similar and I tried to downgrade my kernel to the latest 6.6 release which fixed the issue.
https://github.com/microsoft/mssql-docker/issues/868
It is quite likely the the same root cause is at play here and maybe the people responsible for Cosmos DB could talk to the people responsible for MS SQL to see if it is actually the same problem.
Here is how I downgraded my kernel if anyone needs it:
$ cd /var/cache/pacman/pkg
$ sudo pacman -U linux-6.6.10.arch1-1-x86_64.pkg.tar.zst linux-headers-6.6.10.arch1-1-x86_64.pkg.tar.zst
$ sudo reboot
You might also want to tell pacman to not update those two packages. You can do that by opening /etc/pacman.conf and adding the following line:
IgnorePkg = linux linux-headers
I still had the older package binaries on my system if you don't have them anymore then you will have to figure out how to get them yourself.
This seems to have worked, thanks a lot! Although I always fear that I break something when doing something like this. This should not be needed.
Just want to note that I have the same issue (also on Arch, on kernel 6.7.3). The temporary workaround that szszoke provided got me up and running again (downgrade to kernel 6.6.1). Hope this gets solved soon.
Seeing the same issue on Fedora 39 so its not just specific to Arch.
docker run \
--publish 8081:8081 \
--publish 10250-10255:10250-10255 \
--interactive \
--tty \
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
This is an evaluation version. There are [145] days left in the evaluation period.
Starting
This program has encountered a fatal error and cannot continue running at Tue May 21 14:20:24 2024
The following diagnostic information is available:
Reason: Fatal Signal (0x00000001)
Signal: SIGABRT - Aborted (6)
Stack:
SP IP Function
---------------- ---------------- ----------------
00007f06cbec2260 000055c49b0babfa <unknown>
00007f06cbec32c0 000055c49b0ba5cf <unknown>
00007f06cbec3520 000055c49b0b9a61 <unknown>
00007f06cbec3540 00007f06d7ba1090 killpg+0x40
00007f06cbec3bd0 00007f06d7ba100b gsignal+0xcb
00007f06cbec3cf0 00007f06d7b80859 abort+0x12b
00007f06cbec3e20 000055c49b052332 <unknown>
00007f06cbec3ed0 000055c49b0dbb14 <unknown>
00007f06cbec3ef0 000055c49b10af38 <unknown>
00007f06cbec3fa0 000055c49b10ad1a <unknown>
00007f06cbec4000 000055c49b05e1ba <unknown>
00007f06cbec4080 000055c49b05de0d <unknown>
00007f06cbec4160 000055c49b0d3361 <unknown>
Process: 19 - cosmosdb-emulator
Thread: 126 (application thread 0x1cc)
Instance Id: 86fef1fc-1395-4339-a1d9-121eb39f9a36
Crash Id: e361823e-d63e-4f26-acdc-80ded371315d
Build stamp: (null)
Distribution: Ubuntu 20.04.6 LTS
Processors: 8
Total Memory: 33393545216 bytes
Timestamp: Tue May 21 14:20:24 2024
Last errno: -34938881
Last errno text: Unknown error -34938881
*********** PAL PANIC CORE DUMP GENERATION FAILED **********
Unable to locate handle-crash.sh. Error: File: signals.cpp:483 [Status: 0xC0000034 Object name not found errno = 0x2(2) No such file or directory]
Executing: /usr/local/bin/cosmos/handle-crash.sh with parameters
handle-crash.sh
/usr/local/bin/cosmos/cosmosdb-emulator
19
/usr/local/bin/cosmos
/tmp/cosmos/appdata/log/
86fef1fc-1395-4339-a1d9-121eb39f9a36
e361823e-d63e-4f26-acdc-80ded371315d
*********** PANIC CORE DUMP GENERATION FAILED **********
Attempt to launch handle-crash.sh failed.
This program has encountered a fatal error and cannot continue running at Tue May 21 14:20:24 2024
The following diagnostic information is available:
Reason: Host Extension RTL_ASSERT (0x00000003)
Status: STATUS_WAIT_2 (0x00000002)
Message: !killTheTarget
Stack:
SP IP Function
---------------- ---------------- ----------------
00007f06d5bfbdf0 000055c49b0babfa <unknown>
00007f06d5bfce50 000055c49b0ba5cf <unknown>
00007f06d5bfd0b0 000055c49b024b66 <unknown>
00007f06d5bfd0e0 000055c49b0be0e4 <unknown>
00007f06d5bfe310 000055c49b0bd7a9 <unknown>
00007f06d5bfe440 00007f06d81fc609 start_thread+0xd9
00007f06d5bfe500 00007f06d7c7d353 clone+0x43
Process: 17 - cosmosdb-emulator
Thread: 18
Instance Id: 86fef1fc-1395-4339-a1d9-121eb39f9a36
Crash Id: e361823e-d63e-4f26-acdc-80ded371315d
Build stamp: (null)
Distribution: Ubuntu 20.04.6 LTS
Processors: 8
Total Memory: 33393545216 bytes
Timestamp: Tue May 21 14:20:24 2024
Last errno: 2
Last errno text: No such file or directory
Aborted (core dumped)
To add another data point: Booting into an older kernel works for me (older docker version did not).
On my KDE Neon machine (Ubuntu based) it fails for the "6.8.0-40-generic" kernel but works on the "6.5.0-45-generic" kernel.
The issue is fixed in new release (2.14.20)
Pull the latest image from mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
