Support `--socket-dir` command line argument

[tjhorner]

The dispatcher server accepts a --socket-dir command line argument to let the user customize the location of the unix domain sockets that are created. The dispatcher client already listens to OVERPASS_SOCKET_DIR, so it would be great if dispatcher_start.sh added the appropriate --socket-dir parameter if OVERPASS_SOCKET_DIR is set. I think some additional work would need to be done for the dispatcher_areas supervisord job and here in the entrypoint.

By default, Overpass stores the domain sockets in the database directory, but this can cause problems if the directory is a network share or otherwise does not support unix domain sockets, so specifying a separate directory for those is required.

Thank you for this Docker image!

[BrendenKingTLG]

I attempted to use a volume mount for the database and store the socket file on the container. Everything seems to be functioning.

https://github.com/BrendenKingTLG/overpass-api-k8

[tjhorner]

@BrendenKingTLG If you are using a normal volume mount on your local filesystem then it will work fine. I ran into issues when I mounted a directory over NFS and used that as my volume mount, but sockets don't work with NFS (understandably)

[BrendenKingTLG]

I use a similar setup with AWS EFS. To avoid issues with socket files being shared across instances, I moved the socket file from the DB volume to a path inside the container.

Step 1: Set OVERPASS_SOCKET_DIR

Define an environment variable to set the socket directory. This can be any path within the container. I use /app/socket:

# Dockerfile (line 45)
ENV OVERPASS_RULES_LOAD=1 \
    OVERPASS_USE_AREAS=true \
    OVERPASS_ALLOW_DUPLICATE_QUERIES=no \
    OVERPASS_SOCKET_DIR=/app/socket

Step 2: Start Dispatcher with --socket-dir

Update your startup script to pass the --socket-dir flag when starting the dispatcher. This argument is supported even though it isn’t shown in the help output:

/app/bin/dispatcher --osm-base --socket-dir=/app/socket --meta --db-dir=/app/db &

Reference: dispatcher_server.cc#L264

Step 3: Mount Only the DB Files

Make sure you mount just the DB directory and not the socket path:

volumeMounts:
  - name: db-volume
    mountPath: /app/db
  - name: data-volume
    mountPath: /data

volumes:
  - name: db-volume
    hostPath:
      path: {{ .Values.volumes.dbHostPath }}
      type: Directory

This way, /app/db includes only the database files, while the sockets remain local to the container. I run multiple API instances sharing the same database without socket conflicts. Hope this helps!

[ThomasShih]

@BrendenKingTLG as a data point, the socket dir flag does not work for me. I was able to modify the command to use the flag using this dockerfile below:

FROM wiktorn/overpass-api

# Create a directory for the socket files
RUN mkdir -p /app/socket && chmod og+rwx -R /app/socket

# Modify the last line of /app/bin/dispatcher_start.sh from
# find /db/db -type s -print0 | xargs -0 --no-run-if-empty rm && /app/bin/dispatcher "${DISPATCHER_ARGS[@]}"
# to
# find /db/db -type s -print0 | xargs -0 --no-run-if-empty rm && /app/bin/dispatcher --socket-dir /app/socket "${DISPATCHER_ARGS[@]}"

# Delete the last line
RUN sed -i '$ d' /app/bin/dispatcher_start.sh

# Add the new line with the modified command
RUN echo 'find /db/db -type s -print0 | xargs -0 --no-run-if-empty rm && /app/bin/dispatcher --socket-dir=/app/socket "${DISPATCHER_ARGS[@]}"' >> /app/bin/dispatcher_start.sh

EXPOSE 80
CMD ["/app/docker-entrypoint.sh"]

I know that adding the flag has some effect because before the chmod, I was only getting Permissions denied errors on that directory, seemingly when it was trying to open up a socket.

However, upon fixing that, the errors I'm getting is: runtime error: open64: 2 No such file or directory /db/db//osm3s_osm_base Unix_Socket::7

It seems like while a socket was created in the new dir, there is still sockets expected at the old location. Probablly why the flag isn't public, since it doesn't work to modify all socket variables.