django
Semaphores don't get created on Alpine Linux
Using Alpine Linux 3.6, semaphores don't seem to be getting created in /dev/shm:
# ls -l /dev/shm
total 8
-rw------- 1 root root 20971520 Oct 18 15:34 asgi-groups
-rw------- 1 root root 104857600 Oct 18 15:34 asgi-messages
Using the same versions listed below on Debian Jessie, the semaphores and shared memory appear in /dev/shm:
# ls -l /dev/shm
total 12
-rw------- 1 root root 20971520 Oct 18 15:38 asgi-groups
-rw------- 1 root root 104857600 Oct 18 15:38 asgi-messages
-rw------- 1 root root 32 Oct 18 15:38 sem.asgi-groups
-rw------- 1 root root 32 Oct 18 15:38 sem.asgi-messages
I'm not sure if it's related to the missing semaphore, but the generated value ends up being None. In Debian, I can at least see the value as {}.
Once daphne or a second worker are started, then the value can't be unpickled and results in the stack trace below.
On Alpine:
Python 3.6.1 (default, May 2 2017, 15:16:41)
[GCC 6.3.0] on linux
On Debian:
Python 3.6.3 (default, Oct 10 2017, 02:29:16)
[GCC 4.9.2] on linux
Both:
asgi-ipc==1.4.1
asgiref==1.1.
channels==1.1.8
daphne==1.3.0
Django==1.11.6
posix-ipc==1.0.0
From Alpine:
File "./manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
utility.execute()
File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/usr/lib/python3.6/site-packages/channels/management/commands/runworker.py", line 83, in handle
worker.run()
File "/usr/lib/python3.6/site-packages/channels/worker.py", line 87, in run
channel, content = self.channel_layer.receive_many(channels, block=True)
File "/usr/lib/python3.6/site-packages/asgiref/base_layer.py", line 43, in receive_many
return self.receive(channels, block)
File "/usr/lib/python3.6/site-packages/asgi_ipc/core.py", line 83, in receive
message = self.message_store.pop(channel)
File "/usr/lib/python3.6/site-packages/asgi_ipc/store.py", line 123, in pop
with self.mutate_value() as value:
File "/usr/lib/python3.6/contextlib.py", line 82, in __enter__
return next(self.gen)
File "/usr/lib/python3.6/site-packages/asgi_ipc/store.py", line 57, in mutate_value
value = pickle.load(self.mmap)
_pickle.UnpicklingError: invalid load key, '\xff'
Found the problem. In Alpine, semaphores don't get prefixed with sem., so the path is being overwritten in the constructor when it creates the shared memory. Maybe this is a difference between musl and glibc?
Can something maybe be added to further distinguish the two? Even though it's redundant on glibc-based Linux version, maybe just append '-semaphore' to the end of the path?
Shared memory stuff is so underspecified I'm sure it's just a difference. Fix shouldn't be too hard, but it might have to wait a bit until I get to rewriting the channel layers to be async.
