docker-steam-headless
docker-steam-headless copied to clipboard
Steam-Headless
[Bug]: Two Docker one Host - Mouse shared between Dockers
Describe the Bug
I use two Docker with separated GPUs. The GPU Part works so far very good. But the Problem ist Mouse & Keyboard. If i move the Mouse Docker 1 the mouse moves in Docker2 as well. To be honest, i tried different things but in the end, i dont know where to start to debug. Host ist Debian 11 with a nvidia 4070 ti super and 3090
Same with audio. The audio from Docker1 is on Docker2 and visa versa.
Steps to Reproduce
No response
Expected Behavior
Audio from Docker 1 plays in stream of Docker1 and visa versa
Input from Mouse und Keyboard from Docker1 is containerd in Docker1 and visa versa
Screenshots
No response
Relevant Settings
No response
Version
latest
Platform
xconf.org on the host:
Section "ServerLayout"
Identifier "Layout0"
Screen 0 "Screen0"
InputDevice "Keyboard0" "CoreKeyboard"
InputDevice "Mouse0" "CorePointer"
Option "Xinerama" "0"
EndSection
Section "ServerLayout"
Identifier "Layout1"
Screen 0 "Screen1"
InputDevice "Keyboard1" "CoreKeyboard"
InputDevice "Mouse1" "CorePointer"
Option "Xinerama" "1"
EndSection
Section "Files"
EndSection
Section "InputDevice"
# generated from default
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mouse0"
Option "Emulate3Buttons" "no"
Option "ZAxisMapping" "4 5"
EndSection
Section "InputDevice"
# generated from default
Identifier "Mouse1"
Driver "mouse"
Option "Device" "/dev/input/mouse2"
Option "Protocol" "auto"
Option "ZAxisMapping" "4 5"
EndSection
Section "InputDevice"
# generated from default
Identifier "Keyboard0"
Driver "kbd"
EndSection
Section "InputDevice"
# generated from default
Identifier "Keyboard1"
Driver "kbd"
EndSection
Section "Monitor"
Identifier "Monitor0"
VendorName "Unknown"
ModelName "Unknown"
Option "DPMS"
EndSection
Section "Monitor"
Identifier "Monitor1"
VendorName "Unknown"
ModelName "Unknown"
Option "DPMS"
EndSection
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BusID "PCI:1:0:0"
EndSection
Section "Device"
Identifier "Device1"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BusID "PCI:9:0:0"
EndSection
Section "Screen"
Identifier "Screen0"
Device "Device0"
Monitor "Monitor0"
DefaultDepth 24
SubSection "Display"
Depth 24
EndSubSection
EndSection
Section "Screen"
Identifier "Screen1"
Device "Device1"
Monitor "Monitor1"
DefaultDepth 24
SubSection "Display"
Depth 24
EndSubSection
EndSection
the compose-file Docker1:
version: "3.8"
services:
steam-headless:
image: josh5/steam-headless:latest
restart: unless-stopped
shm_size: ${SHM_SIZE}
#privileged: true
ipc: host # Could also be set to 'shareable'
ulimits:
nofile:
soft: 1024
hard: 524288
cap_add:
- NET_ADMIN
- SYS_ADMIN
- SYS_NICE
security_opt:
- seccomp:unconfined
- apparmor:unconfined
# NETWORK:
## NOTE: With this configuration, if we do not use the host network, then physical device input
## is not possible and your USB connected controllers will not work in steam games.
network_mode: host
hostname: ${NAME}
extra_hosts:
- "${NAME}:127.0.0.1"
# ENVIRONMENT:
## Read all config variables from the .env file
environment:
# System
- TZ=${TZ}
- USER_LOCALES=${USER_LOCALES}
- DISPLAY=${DISPLAY}
# User
- PUID=${PUID}
- PGID=${PGID}
- UMASK=${UMASK}
- USER_PASSWORD=${USER_PASSWORD}
# Mode
- MODE=${MODE}
# Web UI
- WEB_UI_MODE=${WEB_UI_MODE}
- ENABLE_VNC_AUDIO=${ENABLE_VNC_AUDIO}
- PORT_NOVNC_WEB=${PORT_NOVNC_WEB}
- NEKO_NAT1TO1=${NEKO_NAT1TO1}
# Steam
- ENABLE_STEAM=${ENABLE_STEAM}
- STEAM_ARGS=${STEAM_ARGS}
# Sunshine
- ENABLE_SUNSHINE=${ENABLE_SUNSHINE}
- SUNSHINE_USER=${SUNSHINE_USER}
- SUNSHINE_PASS=${SUNSHINE_PASS}
# Xorg
- ENABLE_EVDEV_INPUTS=${ENABLE_EVDEV_INPUTS}
- FORCE_X11_DUMMY_CONFIG=${FORCE_X11_DUMMY_CONFIG}
# Nvidia specific config
- NVIDIA_DRIVER_CAPABILITIES=${NVIDIA_DRIVER_CAPABILITIES}
- NVIDIA_VISIBLE_DEVICES=${NVIDIA_VISIBLE_DEVICES}
- NVIDIA_DRIVER_VERSION=${NVIDIA_DRIVER_VERSION}
runtime: nvidia
# DEVICES:
devices:
# Use the host fuse device [REQUIRED].
- /dev/fuse
# Add the host uinput device [REQUIRED].
- /dev/uinput
# Add NVIDIA HW accelerated devices [OPTIONAL].
# Add NVIDIA HW accelerated devices [OPTIONAL].
# NOTE: If you use the nvidia container toolkit, this is not needed.
# Installing the nvidia container toolkit is the recommended method for running this container
#- /dev/nvidia3
# - /dev/nvidia0
# - /dev/nvidia1
# - /dev/nvidia2
# - /dev/nvidiactl
# - /dev/nvidia-modeset
# - /dev/nvidia-uvm
# - /dev/nvidia-uvm-tools
# - /dev/nvidia-caps/nvidia-cap1
# - /dev/nvidia-caps/nvidia-cap2
# - /dev/dri/
# Ensure container access to devices 13:*
# NOTE: If you use the nvidia container toolkit, this is not needed.
# Installing the nvidia container toolkit is the recommended method for running this container
# GPU PASSTHROUGH
deploy:
resources:
reservations:
# Enable support for NVIDIA GPUs.
#
# Ref: https://docs.docker.com/compose/gpu-support/#enabling-gpu-access-to-service-containers
devices:
- capabilities: [gpu]
device_ids: ["${NVIDIA_VISIBLE_DEVICES}"]
device_cgroup_rules:
- 'c 13:* rmw'
# VOLUMES:
volumes:
# The location of your home directory.
- ${HOME_DIR}/:/home/default/:rw
# The location where all games should be installed.
# This path needs to be set as a library path in Steam after logging in.
# Otherwise, Steam will store games in the home directory above.
- ${GAMES_DIR}/:/mnt/games/:rw
# The Xorg socket.
- ${SHARED_SOCKETS_DIR}/.X11-unix/:/tmp/.X11-unix/:rw
# Pulse audio socket.
- ${SHARED_SOCKETS_DIR}/pulse/:/tmp/pulse/:rw
the .env file Docker1:
NAME=SteamHeadless2
TZ=Pacific/Auckland
USER_LOCALES=en_US.UTF-8 UTF-8
DISPLAY=:1
SHM_SIZE=2G
HOME_DIR=/srv/dev-disk-by-uuid-2ba384f1-3b00-4b0d-8e09-007772e6f1bf/Docker/steamos1/home
SHARED_SOCKETS_DIR=/opt/container-data/steam-headless/sockets
GAMES_DIR=/srv/dev-disk-by-uuid-2ba384f1-3b00-4b0d-8e09-007772e6f1bf/Docker/steamos/games
PUID=1000
PGID=100
UMASK=000
USER_PASSWORD=password
MODE=primary
WEB_UI_MODE=vnc
ENABLE_VNC_AUDIO=false
PORT_NOVNC_WEB=8084
NEKO_NAT1TO1=
ENABLE_STEAM=true
STEAM_ARGS=-silent
ENABLE_SUNSHINE=true
SUNSHINE_USER=admin
SUNSHINE_PASS=admin
ENABLE_EVDEV_INPUTS=true
FORCE_X11_DUMMY_CONFIG=false
NVIDIA_DRIVER_CAPABILITIES=all
NVIDIA_VISIBLE_DEVICES=GPU-4d3a6b72-e370-87f2-721f-428302c1a40d
NVIDIA_DRIVER_VERSION=565.57.01
The docker-compose file Docker 2:
---
version: "3.8"
services:
steam-headless:
image: josh5/steam-headless:latest
restart: unless-stopped
shm_size: ${SHM_SIZE}
#privileged: true
ipc: host # Could also be set to 'shareable'
ulimits:
nofile:
soft: 1024
hard: 524288
cap_add:
- NET_ADMIN
- SYS_ADMIN
- SYS_NICE
security_opt:
- seccomp:unconfined
- apparmor:unconfined
# NETWORK:
## NOTE: With this configuration, if we do not use the host network, then physical device input
## is not possible and your USB connected controllers will not work in steam games.
network_mode: host
hostname: ${NAME}
extra_hosts:
- "${NAME}:127.0.0.1"
# ENVIRONMENT:
## Read all config variables from the .env file
environment:
# System
- TZ=${TZ}
- USER_LOCALES=${USER_LOCALES}
- DISPLAY=${DISPLAY}
# User
- PUID=${PUID}
- PGID=${PGID}
- UMASK=${UMASK}
- USER_PASSWORD=${USER_PASSWORD}
# Mode
- MODE=${MODE}
# Web UI
- WEB_UI_MODE=${WEB_UI_MODE}
- ENABLE_VNC_AUDIO=${ENABLE_VNC_AUDIO}
- PORT_NOVNC_WEB=${PORT_NOVNC_WEB}
- NEKO_NAT1TO1=${NEKO_NAT1TO1}
# Steam
- ENABLE_STEAM=${ENABLE_STEAM}
- STEAM_ARGS=${STEAM_ARGS}
# Sunshine
- ENABLE_SUNSHINE=${ENABLE_SUNSHINE}
- SUNSHINE_USER=${SUNSHINE_USER}
- SUNSHINE_PASS=${SUNSHINE_PASS}
# Xorg
- ENABLE_EVDEV_INPUTS=${ENABLE_EVDEV_INPUTS}
- FORCE_X11_DUMMY_CONFIG=${FORCE_X11_DUMMY_CONFIG}
# Nvidia specific config
- NVIDIA_DRIVER_CAPABILITIES=${NVIDIA_DRIVER_CAPABILITIES}
- NVIDIA_VISIBLE_DEVICES=${NVIDIA_VISIBLE_DEVICES}
- NVIDIA_DRIVER_VERSION=${NVIDIA_DRIVER_VERSION}
runtime: nvidia
# DEVICES:
devices:
# Use the host fuse device [REQUIRED].
- /dev/fuse
# Add the host uinput device [REQUIRED].
- /dev/uinput
# Add NVIDIA HW accelerated devices [OPTIONAL].
# Add NVIDIA HW accelerated devices [OPTIONAL].
# NOTE: If you use the nvidia container toolkit, this is not needed.
# Installing the nvidia container toolkit is the recommended method for running this container
#- /dev/nvidia3
# - /dev/nvidia0
# - /dev/nvidia1
# - /dev/nvidia2
# - /dev/nvidiactl
# - /dev/nvidia-modeset
# - /dev/nvidia-uvm
# - /dev/nvidia-uvm-tools
# - /dev/nvidia-caps/nvidia-cap1
# - /dev/nvidia-caps/nvidia-cap2
# - /dev/dri/
# Ensure container access to devices 13:*
# NOTE: If you use the nvidia container toolkit, this is not needed.
# Installing the nvidia container toolkit is the recommended method for running this container
# GPU PASSTHROUGH
deploy:
resources:
reservations:
# Enable support for NVIDIA GPUs.
#
# Ref: https://docs.docker.com/compose/gpu-support/#enabling-gpu-access-to-service-containers
devices:
- capabilities: [gpu]
device_ids: ["${NVIDIA_VISIBLE_DEVICES}"]
device_cgroup_rules:
- 'c 13:* rmw'
# VOLUMES:
volumes:
# The location of your home directory.
- ${HOME_DIR}/:/home/default/:rw
# The location where all games should be installed.
# This path needs to be set as a library path in Steam after logging in.
# Otherwise, Steam will store games in the home directory above.
- ${GAMES_DIR}/:/mnt/games/:rw
# The Xorg socket.
- ${SHARED_SOCKETS_DIR}/.X11-unix/:/tmp/.X11-unix/:rw
# Pulse audio socket.
- ${SHARED_SOCKETS_DIR}/pulse/:/tmp/pulse/:rw
.env file Docker2:
NAME=SteamHeadless2
TZ=Pacific/Auckland
USER_LOCALES=en_US.UTF-8 UTF-8
DISPLAY=:1
SHM_SIZE=2G
HOME_DIR=/srv/dev-disk-by-uuid-2ba384f1-3b00-4b0d-8e09-007772e6f1bf/Docker/steamos1/home
SHARED_SOCKETS_DIR=/opt/container-data/steam-headless/sockets
GAMES_DIR=/srv/dev-disk-by-uuid-2ba384f1-3b00-4b0d-8e09-007772e6f1bf/Docker/steamos/games
PUID=1000
PGID=100
UMASK=000
USER_PASSWORD=password
MODE=primary
WEB_UI_MODE=vnc
ENABLE_VNC_AUDIO=false
PORT_NOVNC_WEB=8084
NEKO_NAT1TO1=
STEAM_ARGS=-silent
ENABLE_SUNSHINE=true
SUNSHINE_USER=admin
SUNSHINE_PASS=admin
ENABLE_EVDEV_INPUTS=true
FORCE_X11_DUMMY_CONFIG=false
NVIDIA_DRIVER_CAPABILITIES=all
NVIDIA_VISIBLE_DEVICES=GPU-4d3a6b72-e370-87f2-721f-428302c1a40d
NVIDIA_DRIVER_VERSION=565.57.01
Relevant log output
No response
This is an issue with the current implementation of udev inside the container. Something that one day I hope to look at improving. For now, this is a known limitation.
Have you checked out how games-on-whales/wolf handles a fake-udev? That might be an effective solution.
This is an issue with the current implementation of udev inside the container. Something that one day I hope to look at improving. For now, this is a known limitation.
I was this close to success... Wanted to host a "virtual" PC for my friend while he is not present at home, so he could play with me. He would be streaming with moonlight from a container, and i would be sitting at the same PC, just on the physical seat. However, yes, as pointed out, sunshine uses uinput, which ruins all of it, because we dont have a separate seat for udev to pass input devices to. Tried to solve it several times to no avail. Method of doing my thing with virtualgl is even worse. I literally nerd-sniped myself for a whole work day, to find out at the end, that it was not worth it. Ugh.
Edit: nevertheless my sobbing, this project is really cool. I had seen amount of work that went into all of the setup. Its nuts. Hats off to you, Josh!
I noticed that Sunshine joystick passes to the container fully, i do not see ANY trace of it on the host system, but when i inspect container, i see it as event26 device, which is not present on the host. However, mouse and keyboard still "stick" to the seat, being both on host and container as event6 and event8 devices respectfully.
It is possible to make it so that host system would ignore input events from specified devices, leaving them to the container. One could make a patch for the sunshine, which could allow to rename devices that Sunshine uses (at least their product id, for example), in order to allow multiple virtual peripherals to co-exist on one system, which, in turn, would allow multiple remote seats at one rig.
Following udev rules are placed in /lib/udev/rules.d/60-sunshine.rules, it already exists, so i placed it here temporarily. But you certainly should make your own rule in /etc/udev/rules.d/, so that it wont be overwritten.
SUBSYSTEM=="input", ATTRS{id/vendor}=="beef", ATTRS{id/product}=="dead", ATTRS{name}=="Mouse passthrough", ENV{LIBINPUT_IGNORE_DEVICE}="1"
SUBSYSTEM=="input", ATTRS{id/vendor}=="beef", ATTRS{id/product}=="dead", ATTRS{name}=="Keyboard passthrough", ENV{LIBINPUT_IGNORE_DEVICE}="1"
So, yes, in short - i did make it kinda work. Would fully test it later, though.
I dont mind building a custom sunshine for steam headless with any needed patches. I'm keen on this becoming a thing where we can play 2 games on one GPU.
Unfortunately, i tested it today, and found out that dummy X11 screen does not support DRI3, which is a no-go for vulkan rendering. In that case, llvmpipe is used, which is awfully slow. I mean, i wouldve been fine with even 50% perfomance loss, but 1-2 fps is not it. But this is another issue, which is https://github.com/Steam-Headless/docker-steam-headless/issues/168 Do not know how to proceed further, i have too little experience in X11/Wayland multiseat configurations.
@kcalbxof - I've been quietly referencing this thread trying to get two distinct containers running in a fresh Debian install with Docker, a single GPU, and Xorg running on the host.
I suspect using a dummy plug will enable support for DRI3 within the container, since it does not have to create a dummy X11 screen, I'll report back if it works.
https://www.amazon.com/FUERAN-Emulator-Headless-Supports-3840x2160/dp/B0D4Z7MR9G
@cwilby i actually have two of those plugs, but i trashed whole docker setup due to having to clean up the disk. Might get back to this a bit later.
@kcalbxof - I've been quietly referencing this thread trying to get two distinct containers running in a fresh Debian install with Docker, a single GPU, and Xorg running on the host.
I suspect using a dummy plug will enable support for DRI3 within the container, since it does not have to create a dummy X11 screen, I'll report back if it works.
https://www.amazon.com/FUERAN-Emulator-Headless-Supports-3840x2160/dp/B0D4Z7MR9G
Steam‑Headless is a great project, but for a multi‑user game platform /parallel game-streamingI strongly recommend using games-on-whales. l
Bear with me for I am but a slightly evolved monkey with access to technology..
Following on from what others have said here:
- Games on Whales is built on Wolf.
- Wolf is a wrapper(?) around Sunshine
- Wolf adds fakeudev (as a ghost user mentioned).
- Their fakeudev implementation creates a socket for each session here
- Our dumbudev implementation relays kernel events through bind mounting(?)
Perhaps dumbudev can be modified or swapped for fakeudev given it's MIT licensing?
Bear with me for I am but a slightly evolved monkey with access to technology..
Following on from what others have said here:
- Games on Whales is built on Wolf.
- Wolf is a wrapper(?) around Sunshine
- Wolf adds fakeudev (as a ghost user mentioned).
- Their fakeudev implementation creates a socket for each session here
- Our dumbudev implementation relays kernel events through bind mounting(?)
Perhaps dumbudev can be modified or swapped for fakeudev given it's MIT licensing?
Wolf is not a sunshine wrapper. It is a complete ground up replacement. Wolf has full control over IO in a container with fakeudev. The fakeudev is for faking udev stuff to apps from wolf, it does not do anything with the kenel messages as far as I am aware. My dumbudev listens for real kernel messages and relays them to the unprivileged container.
It is possible to have sunshine manage input without udev and uinput, but we would need to patch it, and I have not currently got the time.