watchfiles Detect WSL

Detect WSL

Open samuelcolvin opened this issue 1 year ago • 9 comments

Update: docker check is not required, see discussion below

and default to polling automatically.

See:

#134
#169
https://github.com/encode/uvicorn/pull/1647

Looks like it shouldn't be too hard to detect WSL, but does this work inside docker?

Detecting docker looks a bit more involved but still doable.

Would be great if some who has access to WSL would work on this, or at the very least agree to test it.

Sep 15 '22 16:09 samuelcolvin

I'm not sure I have the skills (or the time) to make the fix, but I'm definitely up for testing it. I have easy access to WSL+docker.

Sep 15 '22 18:09 danroozemond

Perhaps as input this helps; note the first stackoverflow solution returns 'True' on WSL, even if not in docker (I do obviously have docker installed, perhaps it's fooled by that) [edited to add cat /proc/self/cgroup]

Windows:

C:\Users\Dan Roozemond>python
Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, platform
>>> sys.version
'3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]'
>>> platform.uname()
uname_result(system='Windows', node='DESKTOP-xxxxxx', release='10', version='10.0.22000', machine='AMD64', processor='Intel64 Family 6 Model 142 Stepping 12, GenuineIntel')
>>> import os
>>> def is_docker():
...     path = '/proc/self/cgroup'
...     return (
...         os.path.exists('/.dockerenv') or
...         os.path.isfile(path) and any('docker' in line for line in open(path))
...     )
...
>>> is_docker()
False
>>> import os, re
>>>
>>> path = "/proc/self/cgroup"
>>>
>>> def is_docker():
...   if not os.path.isfile(path): return False
...   with open(path) as f:
...     for line in f:
...       if re.match("\d+:[\w=]+:/docker(-[ce]e)?/\w+", line):
...         return True
...     return False
...
>>> print(is_docker())
False

Docker + WSL:

C:\Users\Dan Roozemond>docker exec -it e4f13 /bin/bash
root@e4f133540f3b:/# python
Python 3.9.14 (main, Sep 13 2022, 14:57:28)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, platform
>>> sys.version
'3.9.14 (main, Sep 13 2022, 14:57:28) \n[GCC 8.3.0]'
>>> platform.uname()
uname_result(system='Linux', node='e4f133540f3b', release='4.19.128-microsoft-standard', version='#1 SMP Tue Jun 23 12:58:10 UTC 2020', machine='x86_64')
>>> import os
>>> def is_docker():
...     path = '/proc/self/cgroup'
...     return (
...         os.path.exists('/.dockerenv') or
...         os.path.isfile(path) and any('docker' in line for line in open(path))
...     )
...
>>> is_docker()
True
>>> import os, re
>>>
>>> path = "/proc/self/cgroup"
>>>
>>> def is_docker():
...   if not os.path.isfile(path): return False
...   with open(path) as f:
...     for line in f:
...       if re.match("\d+:[\w=]+:/docker(-[ce]e)?/\w+", line):
...         return True
...     return False
...
>>> print(is_docker())
True
root@e4f133540f3b:/# cat /proc/self/cgroup
38:name=systemd:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
37:rdma:/
36:pids:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
35:hugetlb:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
34:net_prio:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
33:perf_event:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
32:net_cls:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
31:freezer:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
30:devices:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
29:blkio:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
28:cpuacct:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
27:cpu:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
26:cpuset:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
1:memory:/docker/e4f133540f3b63251c34839a46ddf1400d3be947e0bc6c72c2ea3a8a8b7d52eb
0::/

WSL, no Docker:

root@DESKTOP-xxxxxx [ ~ ]# python3
Python 3.7.13 (default, Jul  9 2022, 10:35:55)
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, platform
>>> sys.version
'3.7.13 (default, Jul  9 2022, 10:35:55) \n[GCC 9.1.0]'
>>> platform.uname()
uname_result(system='Linux', node='DESKTOP-xxxxxx', release='5.10.102.1-microsoft-standard-WSL2', version='#1 SMP Wed Mar 2 00:30:59 UTC 2022', machine='x86_64', processor='x86_64')
>>> import os
>>> def is_docker():
...     path = '/proc/self/cgroup'
...     return (
...         os.path.exists('/.dockerenv') or
...         os.path.isfile(path) and any('docker' in line for line in open(path))
...     )
...
>>> is_docker()
True
>>> import os, re
>>>
>>> path = "/proc/self/cgroup"
>>>
>>> def is_docker():
...   if not os.path.isfile(path): return False
...   with open(path) as f:
...     for line in f:
...       if re.match("\d+:[\w=]+:/docker(-[ce]e)?/\w+", line):
...         return True
...     return False
...
>>> print(is_docker())
False
>>>
root@DESKTOP-xxxxxx [ ~ ]# cat /proc/self/cgroup
38:name=systemd:/
37:rdma:/
36:pids:/
35:hugetlb:/
34:net_prio:/
33:perf_event:/
32:net_cls:/
31:freezer:/
30:devices:/
29:blkio:/
28:cpuacct:/
27:cpu:/
26:cpuset:/
1:memory:/
0::/

Sep 15 '22 18:09 danroozemond

It seems the first variant returns True because of the existence of /.dockerenv, while that's mostly an indicator that docker is installed, rather than that we're currently running in docker.

Sep 15 '22 18:09 danroozemond

How about we default to polling if the following returns true?

from pathlib import Path
import platform
import re


def auto_poll() -> bool:
    if 'microsoft-standard' not in platform.uname().release:
        return False

    cgroup_path = Path('/proc/self/cgroup')

    if cgroup_path.is_file():
        text = cgroup_path.read_bytes()
        return bool(re.match(br'\d+:[\w=]+:/docker(-[ce]e)?/\w+', text))
    else:
        return False

That could (I think) lead to polling in some scenarios where it's not actually required, but:

In most scenarios polling is fine anyway.
The user can always manually deactivate polling (we might need another env. variable for this)

@danroozemond, any chance you could try that function and let me know if it performs correctly?

Sep 16 '22 09:09 samuelcolvin

To be clear, the problem AFAIR is not Docker at all, it's when you are running under WSL and polling a Windows drive mounted via the cross-mount support in WSL (Plan9-based) per https://github.com/microsoft/WSL/issues/4739.

So the most-common case is probably a host-mounted Windows path inside a Docker Desktop WSL integration-driven Linux container, but we don't need to poll if the directory being polled is not a Windows host-mount (e.g. a Docker volume would be fine, or a tmpfs), and we do need to poll even if not using Docker, as long as it's WSL accessing a Windows host mount.

This latter case is one of the two failed attempts in the #169 original report.

In plain WSL, the mount-type of affected drives is 9p, and this also appears to be the case inside Docker Desktop's WSL support. In the latter case, the source path of the mount is fake, I'm not sure how exactly it handles this underneath. I expected it to be a bind-mount, personally, so this is an easier-to-detect result.

You might be able to shortcut this by noting that the mounted device appears to be a Windows drive letter in both cases, but that might be a sampling error.

Dump of `mount` run inside WSL Ubuntu

paulh@KEITARO:~$ mount
/dev/sdb on / type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
tmpfs on /mnt/wsl type tmpfs (rw,relatime)
tools on /init type 9p (ro,relatime,dirsync,aname=tools;fmask=022,loose,access=client,trans=fd,rfd=6,wfd=6)
none on /dev type devtmpfs (rw,nosuid,relatime,size=13109396k,nr_inodes=3277349,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
devpts on /dev/pts type devpts (rw,nosuid,noexec,noatime,gid=5,mode=620,ptmxmode=000)
none on /run type tmpfs (rw,nosuid,noexec,noatime,mode=755)
none on /run/lock type tmpfs (rw,nosuid,nodev,noexec,noatime)
none on /run/shm type tmpfs (rw,nosuid,nodev,noatime)
none on /run/user type tmpfs (rw,nosuid,nodev,noexec,noatime,mode=755)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755)
drivers on /usr/lib/wsl/drivers type 9p (ro,nosuid,nodev,noatime,dirsync,aname=drivers;fmask=222;dmask=222,mmap,access=client,msize=65536,trans=fd,rfd=4,wfd=4)
lib on /usr/lib/wsl/lib type 9p (ro,nosuid,nodev,noatime,dirsync,aname=lib;fmask=222;dmask=222,mmap,access=client,msize=65536,trans=fd,rfd=4,wfd=4)
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu type cgroup (rw,nosuid,nodev,noexec,relatime,cpu)
cgroup on /sys/fs/cgroup/cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)
C:\ on /mnt/c type 9p (rw,noatime,dirsync,aname=drvfs;path=C:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=65536,trans=fd,rfd=8,wfd=8)
D:\ on /mnt/d type 9p (rw,noatime,dirsync,aname=drvfs;path=D:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=65536,trans=fd,rfd=8,wfd=8)
E:\ on /mnt/e type 9p (rw,noatime,dirsync,aname=drvfs;path=E:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=65536,trans=fd,rfd=8,wfd=8)

Dump of `mount` from a busybox container from Docker Desktop's Linux support

> docker run --rm -it --mount type=bind,source="${PWD}",target=/app busybox mount
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/TVEN2ZOQZ2N2LFMTBXO2U2SQ5T:/var/lib/docker/overlay2/l/2WKN5YQVDV5B4SCWPY37EYUVDN,upperdir=/var/lib/docker/overlay2/37f16f66163da66a1de222e392cc3ce88abfe1e6b53eac418f4430717f4acc60/diff,workdir=/var/lib/docker/overlay2/37f16f66163da66a1de222e392cc3ce88abfe1e6b53eac418f4430717f4acc60/work)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev type tmpfs (rw,nosuid,size=65536k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755)
cpuset on /sys/fs/cgroup/cpuset type cgroup (ro,nosuid,nodev,noexec,relatime,cpuset)
cpu on /sys/fs/cgroup/cpu type cgroup (ro,nosuid,nodev,noexec,relatime,cpu)
cpuacct on /sys/fs/cgroup/cpuacct type cgroup (ro,nosuid,nodev,noexec,relatime,cpuacct)
blkio on /sys/fs/cgroup/blkio type cgroup (ro,nosuid,nodev,noexec,relatime,blkio)
memory on /sys/fs/cgroup/memory type cgroup (ro,nosuid,nodev,noexec,relatime,memory)
devices on /sys/fs/cgroup/devices type cgroup (ro,nosuid,nodev,noexec,relatime,devices)
freezer on /sys/fs/cgroup/freezer type cgroup (ro,nosuid,nodev,noexec,relatime,freezer)
net_cls on /sys/fs/cgroup/net_cls type cgroup (ro,nosuid,nodev,noexec,relatime,net_cls)
perf_event on /sys/fs/cgroup/perf_event type cgroup (ro,nosuid,nodev,noexec,relatime,perf_event)
net_prio on /sys/fs/cgroup/net_prio type cgroup (ro,nosuid,nodev,noexec,relatime,net_prio)
hugetlb on /sys/fs/cgroup/hugetlb type cgroup (ro,nosuid,nodev,noexec,relatime,hugetlb)
pids on /sys/fs/cgroup/pids type cgroup (ro,nosuid,nodev,noexec,relatime,pids)
rdma on /sys/fs/cgroup/rdma type cgroup (ro,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/systemd type cgroup (ro,nosuid,nodev,noexec,relatime,name=systemd)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
C:\134 on /app type 9p (rw,dirsync,noatime,aname=drvfs;path=C:\;uid=0;gid=0;metadata;symlinkroot=/mnt/host,mmap,access=client,msize=65536,trans=fd,rfd=8,wfd=8)
/dev/sdd on /etc/resolv.conf type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
/dev/sdd on /etc/hostname type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
/dev/sdd on /etc/hosts type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
proc on /proc/bus type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/fs type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/irq type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
tmpfs on /proc/acpi type tmpfs (ro,relatime)
tmpfs on /proc/kcore type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/keys type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/timer_list type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/sched_debug type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /sys/firmware type tmpfs (ro,relatime)

Dump of `mount` from a busybox container from Docker Desktop's Linux support run inside WSL Ubuntu

paulh@KEITARO:~$ docker run --rm -it --mount type=bind,source=/mnt/c/Users/paulh,target=/app busybox mount
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/C4FHANTRRPYU55VQHDYQRM3LZH:/var/lib/docker/overlay2/l/2WKN5YQVDV5B4SCWPY37EYUVDN,upperdir=/var/lib/docker/overlay2/46aecf00b13fdc601bee2eab2c474ade54ebdeaa2cf94c3b9cfc2584bf7c61d5/diff,workdir=/var/lib/docker/overlay2/46aecf00b13fdc601bee2eab2c474ade54ebdeaa2cf94c3b9cfc2584bf7c61d5/work)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev type tmpfs (rw,nosuid,size=65536k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755)
cpuset on /sys/fs/cgroup/cpuset type cgroup (ro,nosuid,nodev,noexec,relatime,cpuset)
cpu on /sys/fs/cgroup/cpu type cgroup (ro,nosuid,nodev,noexec,relatime,cpu)
cpuacct on /sys/fs/cgroup/cpuacct type cgroup (ro,nosuid,nodev,noexec,relatime,cpuacct)
blkio on /sys/fs/cgroup/blkio type cgroup (ro,nosuid,nodev,noexec,relatime,blkio)
memory on /sys/fs/cgroup/memory type cgroup (ro,nosuid,nodev,noexec,relatime,memory)
devices on /sys/fs/cgroup/devices type cgroup (ro,nosuid,nodev,noexec,relatime,devices)
freezer on /sys/fs/cgroup/freezer type cgroup (ro,nosuid,nodev,noexec,relatime,freezer)
net_cls on /sys/fs/cgroup/net_cls type cgroup (ro,nosuid,nodev,noexec,relatime,net_cls)
perf_event on /sys/fs/cgroup/perf_event type cgroup (ro,nosuid,nodev,noexec,relatime,perf_event)
net_prio on /sys/fs/cgroup/net_prio type cgroup (ro,nosuid,nodev,noexec,relatime,net_prio)
hugetlb on /sys/fs/cgroup/hugetlb type cgroup (ro,nosuid,nodev,noexec,relatime,hugetlb)
pids on /sys/fs/cgroup/pids type cgroup (ro,nosuid,nodev,noexec,relatime,pids)
rdma on /sys/fs/cgroup/rdma type cgroup (ro,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/systemd type cgroup (ro,nosuid,nodev,noexec,relatime,name=systemd)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
C:\134 on /app type 9p (rw,dirsync,noatime,aname=drvfs;path=C:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=65536,trans=fd,rfd=8,wfd=8)
/dev/sdd on /etc/resolv.conf type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
/dev/sdd on /etc/hostname type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
/dev/sdd on /etc/hosts type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
proc on /proc/bus type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/fs type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/irq type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
tmpfs on /proc/acpi type tmpfs (ro,relatime)
tmpfs on /proc/kcore type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/keys type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/timer_list type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/sched_debug type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /sys/firmware type tmpfs (ro,relatime)

Sep 16 '22 11:09 TBBle

Sounds like the simplest solution might be just:

import platform

def auto_poll() -> bool:
    return 'microsoft-standard' in platform.uname().release

WDYT?

I favour this as it's much less complex than mount checks and should be easy to reason with for end users.

Sep 16 '22 11:09 samuelcolvin

or 'microsoft-standard' in uname.release.lower() and uname.system.lower() == 'linux'?

Sep 16 '22 11:09 samuelcolvin

Yeah, as long as there's a way to override that detection when it's known to not be monitoring a 9p filesystem, then it's a fail-safe default since inotify doesn't fail on WSL 9p mounts, just silently produces no results.

Sep 16 '22 11:09 TBBle

Agreed, _default_force_pulling should only check for WSL if force_polling is None, same as it currently only respected WATCHFILES_FORCE_POLLING when force_polling is None, so when using watchfiles directly, that should already work.

https://github.com/samuelcolvin/watchfiles/blob/32c0f331a73109859ed655fbcf38beec19842647/watchfiles/main.py#L292-L299

But for the (most common) case where somones is using watchfiles from another library like arq or uvicorn, we need to modify the logic such that

WATCHFILES_FORCE_POLLING=false
# or 
WATCHFILES_FORCE_POLLING=disable

Specifically mean don't force polling even on WSL.

While making this change, we should also follow the convention that an empty env var is equivilant to not setting it.

PR welcome, otherwise I'll get to this when I can.

Sep 16 '22 11:09 samuelcolvin

This is (hopefully) fixed in #194, please can someone with windows and WSL (and ideally docker) run watchfiles from that branch and check it's working correctly?

Oct 06 '22 10:10 samuelcolvin

watchfiles watchfiles copied to clipboard

Detect WSL

watchfiles
watchfiles copied to clipboard