`FileNotFoundError` on files on `reload-exclude`

[Kludex]

When I use --reload-exclude, or another directory is supposed to be watched, I still have the following:

Running development server on 0.0.0.0:8000
INFO:     Will watch for changes in these directories: ['/app/src']
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [7] using WatchFiles
Traceback (most recent call last):
  File "/opt/app-env/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/main.py", line 408, in main
    run(
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/main.py", line 571, in run
    ChangeReload(config, target=server.run, sockets=[sock]).run()
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/basereload.py", line 45, in run
    for changes in self:
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/basereload.py", line 64, in __next__
    return self.should_restart()
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/watchfilesreload.py", line 85, in should_restart
    changes = next(self.watcher)
  File "/opt/app-env/lib/python3.8/site-packages/watchfiles/main.py", line 119, in watch
    with RustNotify([str(p) for p in paths], debug, force_polling, poll_delay_ms, recursive) as watcher:
FileNotFoundError: Permission denied (os error 13) about ["/app/data"]

The command used was:

uvicorn --reload --reload-dir /app/src main:app

I guess it's because we don't really exclude, we just check the path to see if we should reload.

cc @samuelcolvin

[!IMPORTANT]

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[samuelcolvin]

yes, notify, and therefore watchfiles doesn't allow you to exclude a directory, just select multiple directories to watch.

[Kludex]

If I wrap the following line on except FileNotFoundError:

  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/watchfilesreload.py", line 85, in should_restart
    changes = next(self.watcher)

Would it be a bad idea? How do you think we should solve this? :thinking:

[samuelcolvin]

I thought about that, but I think the rust code will exit early if it finds an error, so you might miss changes. I haven't checked, that's just my guess.

[samuelcolvin]

you might also ignore other errors, not just permission denied.

[Kludex]

Can I interpret the above as "on uvicorn, we should instead wait for https://github.com/samuelcolvin/watchfiles/issues/147, and then change the behavior to 'ignore these errors'"?

[samuelcolvin]

yes.

[Kludex]

Thanks 🙏

[AdamMusa]

I resolved this problem by typing this command in my terminal sudo sysctl -w fs.inotify.max_user_watches=457856

[aminalaee]

@Kludex I have merged https://github.com/samuelcolvin/watchfiles/issues/147 Should we just add a new flag ignore-permission-denied here? Or do we want to implicitly enable that when we have reload_excludes? If we can decide it I will continue the work.

[samuelcolvin]

You can just set the env variable. I think that's enough for now.

[Yasamato]

@samuelcolvin could you refer me to which env variable you are talking?

I am currently experiencing the same problem of a sub directory having restricted permissions and crashing the --reload functionality.

For replication:

pip install uvicorn[standard]

docker-compose.yml

version: '3'
services:
  db:
    image: mariadb
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: mydatabase
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    volumes:
      - ./db:/var/lib/mysql

and start with docker compose up -d creating the folder ./db. Now we have some python web app in ./api and run it via uvicorn:

uvicorn api.main:app --reload  --port 3000 --reload-dir api --reload-exclude 'db/**/*'

But this results in original posted permissions denied error and the functionality of hot reloading not working.