Question: How to reload when a SASS file changes?

[vitorbaptista]

First of all, thanks for your work. It has been helping me immensely.

I have the livesync working fine on Django 2.1.1, but now I want to use SASS. To do so, I'm using https://github.com/jrief/django-sass-processor. The problem is that when the SASS files are modified, there's no automatic reloads. Is there any way to make livesync monitor *.scss files as well?

(If you prefer to answer on SO, I asked the same question on https://stackoverflow.com/questions/52167436/how-to-automatically-reload-a-django-app-using-django-livesync-when-a-scss-file)

[fabiogibson]

Hi Vitor,

It's good to hear that livesync has been useful for you.

I'll check how the sass-processor works. For now, do you mind cloning livesync, include *.scss in livesync/core/handler.py and install from source (pip install -e .) inside a virtualenv to check if it works?

I will have time to check that at night and problably create a new setting for extending watched extensions.

[vitorbaptista]

Hey @fabiogibson, thanks for the quick reply.

I tried that change. Sometimes it doesn't work (nothing happens at all when I change the scss), other times it keeps reloading the page. I couldn't figure out what caused the different results.

If it helps, the Django project I'm running is http://github.com/vitorbaptista/lainonima. Although the README is in Portuguese, I'm sure you won't have issues running it.

[fabiogibson]

@vitorbaptista I'll work on that and let you know asap.

[fabiogibson]

Hi @vitorbaptista, the problem happens because django-sass-processor only compiles scss files during requests, so when django-livesync watches for css changes, nothing happens. In the other side, when watching for scss files, the reload process is correctly requested to sync-server but it also triggers the sass-processor compilation which then causes an unexpected request to sync-server. I'll try to figure out a way to handle it.

[vitorbaptista]

@fabiogibson I spent a few hours debugging this issue, but couldn't find a good solution. However, I think I found the problem: when updating the CSS, django-sass-processor deletes and recreates the file.

When the CSS is deleted, it is removed from the self.history dict. Then, when it's created again, the on_modified() method checks its md5 against what it has in self.history, which now doesn't contain the MD5, so a reload is triggered. This creates the infinite loop.

At first I tried not removing the MD5 from the self.history when a file is deleted (which would cause other problems, but I was just trying things out). It still doesn't solve the issue, because even if we keep the history, we need to reload the page after the deletion, which causes django-sass-processor to delete the file again, and we're back in the infinite loop.

I went on to see the code on django-sass-processor and the deletion happens on https://github.com/jrief/django-sass-processor/blob/78c8dd1511ed82a23cb7fcc6bf29e7b88ed72ed5/sass_processor/processor.py#L107-L108. I commented out these lines, but then a few temporary files started appearing in my staticfiles dir (e.g. main_xaczf.css). I couldn't find where those are created.

The solution seems to be:

1. Change django-sass-processor to not delete the CSS files, only overwrite them
2. Monitor SCSS files as well (so we reload the page and trigger the CSS compilation, this will cause SCSS changes to reload the page twice)

Things ended up being more complex than I expected. Although it would be great to use django-sass-processor, I'll try to remove it, as I imagine the fix to this issue will take some time :/

[fabiogibson]

Hi @vitorbaptista, I think I’ve got it working with a new approach. LiveSync is not watching for the root directory anymore and doesn’t care about file extensions. Instead, it’s looking for static files and template dirs, based on the project settings. This way any complied file can be copied to the static root folder (which is not being watched) without triggering multiple requests or infinite loops. Any file extension including scss will be natively handled since it’s located in a static folder, so we can also benefit from the live refresh when replacing images or any kind of asset. The change is on branch 0.5, not on pypi yet.

Would appreciate if you have some time to give it a try and provide some feedback.

[vitorbaptista]

Hey @fabiogibson, that's a great approach! Unfortunately, I get a strange error when trying out your new branch: if I change any file, it tries reloading an inexistent file 4913 in the same folder. For instance, I changed web/static/web/styles/main.scss and this is what I got:

on_modified <FileModifiedEvent: src_path='/home/vitor/Projetos/lainonima/web/static/web/styles/4913'>
update_history /home/vitor/Projetos/lainonima/web/static/web/styles/4913
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/vitor/.pyenv/versions/3.6.3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/vitor/Projetos/lainonima/env/lib/python3.6/site-packages/watchdog/observers/api.py", line 199, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/home/vitor/Projetos/lainonima/env/lib/python3.6/site-packages/watchdog/observers/api.py", line 368, in dispatch_events
    handler.dispatch(event)
  File "/home/vitor/Projetos/lainonima/env/lib/python3.6/site-packages/watchdog/events.py", line 454, in dispatch
    _method_map[event_type](event)
  File "/home/vitor/Projetos/lainonima/env/lib/python3.6/site-packages/livesync/fswatcher/handlers.py", line 45, in on_modified
    if self._update_history(event.src_path):
  File "/home/vitor/Projetos/lainonima/env/lib/python3.6/site-packages/livesync/fswatcher/handlers.py", line 24, in _update_history
    md5hash = get_md5(path)
  File "/home/vitor/Projetos/lainonima/env/lib/python3.6/site-packages/livesync/fswatcher/utils.py", line 7, in get_md5
    with open(fname, "rb") as rfile:
FileNotFoundError: [Errno 2] No such file or directory: '/home/vitor/Projetos/lainonima/web/static/web/styles/4913'

(There are some debug messages in there)

You can reproduce it easily, just:

1. Clone and install http://github.com/vitorbaptista/lainonima
2. git checkout 5e7717d7ca10e7f982d629069d2fd95b72ddb397 to get back to a version with django-sass
3. pip install git+https://github.com/fabiogibson/[email protected]
4. python manage.py runserver
5. Change any file (e.g. web/static/web/styles/main.scss) and see what happens

[fabiogibson]

Hi @vitorbaptista,

First of all, thanks for your help!

I've just created an empty virtualenv from your requirements.txt and setup lainonima from scratch following the steps you provided but could not reproduce the error. Editing main.scss created css and css.map files at 'staticfiles' folder and triggered the refresh normally.

Can you check which version of watchdog was installed on your virtualenv?

Are you facing this issue only with scss files or with every file when using 0.5 branch?

[vitorbaptista]

@fabiogibson You are right, I tried cloning my repository in a different folder and everything seems to work fine. There must be some file somewhere causing havoc, although I can't think of what would cause such strange behaviour.

[src-r-r]

There seems to be an issue either with watchdog or livesync:

    Traceback (most recent call last):
  File "./manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 61, in execute
    super(Command, self).execute(*args, **options)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "$LIVESYNC_DIR/livesync/management/commands/runserver.py", line 110, in handle
    self.start_liveserver(**options)
  File "$LIVESYNC_DIR/livesync/management/commands/runserver.py", line 107, in start_liveserver
    self._start_watchdog()
  File "$LIVESYNC_DIR/livesync/management/commands/runserver.py", line 81, in _start_watchdog
    self.file_watcher.start()
  File "$LIVESYNC_DIR/livesync/fswatcher/watcher.py", line 19, in start
    self.observer.start()
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/watchdog/observers/api.py", line 255, in start
    emitter.start()
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 110, in start
    self.on_thread_start()
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/watchdog/observers/inotify.py", line 121, in on_thread_start
    self._inotify = InotifyBuffer(path, self.watch.is_recursive)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/watchdog/observers/inotify_buffer.py", line 35, in __init__
    self._inotify = Inotify(path, recursive)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 199, in __init__
    self._add_dir_watch(path, recursive, event_mask)
  File "$VIRTUALENV_DIR/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 379, in _add_dir_watch
    raise OSError('Path is not a directory')
OSError: Path is not a directory

"Well, thanks inotify," I said. "It might help to know what path is not a directory..."

In any case, if I add the following snippet to livesync/fs/watcher.py (importing os, of course):

    def _schedule_all(self):
        for handler in self.handlers:
            for path in handler.watched_paths:
                path = os.path.abspath(path)
                if not os.path.exists(path):
                    print("Not adding path {}".format(path))
                    continue
                print("adding path {}".format(path))
                self.observer.schedule(handler, path, recursive=True)

I get the following printed to the console:

adding path <static_path_1>
adding path <static_path_2>
...
Not adding path <project_base_dir>/templates
...
adding path <static_path_1>

And, indeed, <project_base_dir>/templates does not exist, but that's the only one.

My hunch is that the issue is actually with watchdog, but do you think it would be appropriate to add this code snippet (minus the prints, of course) so that we can ignore files that don't exist? In fact, it might be a good idea to add a LIVESYNC_HARD_FAIL setting or something to allow users to decide for themselves.