HABApp
HABApp copied to clipboard
spacemanspiff2007
Filewatcher problem with rsync
Hi I use rsync to deploy configuration to production, and have habapp running in a docker container. Recently I found a strange behavior of filewatcher. After copying a new file using rsync, it removes any references to it. Probably it happens because rsync creates temporary file and then moves it to final location. Habapp sees this move and then interprets it as file removal. EDIT: Probably the answer is here: https://github.com/spacemanspiff2007/HABApp/blob/5ccb2d6f69e7917a766cac60ac25db5afa2919e6/src/HABApp/core/files/watcher/base_watcher.py#L55 and here: https://github.com/spacemanspiff2007/HABApp/blob/5ccb2d6f69e7917a766cac60ac25db5afa2919e6/src/HABApp/core/files/watcher/base_watcher.py#L61
You can see it in the logs (it's Loki, read from bottom):
| | 2024-03-13 12:02:39.232 | [2024-03-13 12:02:39,232] [ HABApp.files] DEBUG \| Removed rules/presence/leave_arrival.py |
-- | -- | -- | -- | --
| | 2024-03-13 12:02:39.231 | [2024-03-13 12:02:39,231] [ HABApp.files] DEBUG \| rules/presence/leave_arrival.py changed to REMOVED |
| | 2024-03-13 12:02:39.229 | [2024-03-13 12:02:39,228] [ HABApp.Rules] DEBUG \| File rules/presence/leave_arrival.py successfully unloaded! |
| | 2024-03-13 12:02:39.229 | [2024-03-13 12:02:39,228] [ HABApp] DEBUG \| Removed event listener for "presence_devices" (filter=ValueChangeEventFilter(value=OPEN)) |
| | 2024-03-13 12:02:39.228 | [2024-03-13 12:02:39,228] [ HABApp] DEBUG \| Removed event listener for "presence_manual" (filter=ValueChangeEventFilter(value=OFF)) |
| | 2024-03-13 12:02:39.228 | [2024-03-13 12:02:39,228] [ HABApp] DEBUG \| Removed event listener for "presence_devices" (filter=ValueChangeEventFilter(value=CLOSED)) |
| | 2024-03-13 12:02:39.228 | [2024-03-13 12:02:39,227] [ HABApp] DEBUG \| Removed event listener for "presence_manual" (filter=ValueChangeEventFilter(value=ON)) |
| | 2024-03-13 12:02:39.228 | [2024-03-13 12:02:39,227] [ HABApp] DEBUG \| Removed event listener for "presence_kids" (filter=ValueChangeEventFilter()) |
| | 2024-03-13 12:02:39.228 | [2024-03-13 12:02:39,227] [ HABApp] DEBUG \| Removed event listener for "presence_devices" (filter=ValueChangeEventFilter(value=OPEN)) |
| | 2024-03-13 12:02:39.223 | [2024-03-13 12:02:39,223] [ HABApp.Rules] DEBUG \| Removing file: rules/presence/leave_arrival.py |
| | 2024-03-13 12:02:38.618 | [2024-03-13 12:02:38,617] [ HABApp.file.events] DEBUG \| DirModifiedEvent(src_path='/habapp/config/rules/presence', dest_path='', event_type='modified', is_directory=True, is_synthetic=False) |
| | 2024-03-13 12:02:38.616 | [2024-03-13 12:02:38,616] [ HABApp.file.events] DEBUG \| FileMovedEvent(src_path='/habapp/config/rules/presence/.leave_arrival.py.1upIJ8', dest_path='/habapp/config/rules/presence/leave_arrival.py', event_type='moved', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.616 | [2024-03-13 12:02:38,616] [ HABApp.file.events] DEBUG \| FileModifiedEvent(src_path='/habapp/config/rules/presence/.leave_arrival.py.1upIJ8', dest_path='', event_type='modified', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.616 | [2024-03-13 12:02:38,615] [ HABApp.file.events] DEBUG \| DirModifiedEvent(src_path='/habapp/config/rules/presence', dest_path='', event_type='modified', is_directory=True, is_synthetic=False) |
| | 2024-03-13 12:02:38.615 | [2024-03-13 12:02:38,615] [ HABApp.file.events] DEBUG \| FileClosedEvent(src_path='/habapp/config/rules/presence/.leave_arrival.py.1upIJ8', dest_path='', event_type='closed', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.615 | [2024-03-13 12:02:38,615] [ HABApp.file.events] DEBUG \| FileModifiedEvent(src_path='/habapp/config/rules/presence/.leave_arrival.py.1upIJ8', dest_path='', event_type='modified', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.615 | [2024-03-13 12:02:38,614] [ HABApp.file.events] DEBUG \| FileOpenedEvent(src_path='/habapp/config/rules/presence/.leave_arrival.py.1upIJ8', dest_path='', event_type='opened', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.614 | [2024-03-13 12:02:38,614] [ HABApp.file.events] DEBUG \| DirModifiedEvent(src_path='/habapp/config/rules/presence', dest_path='', event_type='modified', is_directory=True, is_synthetic=False) |
| | 2024-03-13 12:02:38.614 | [2024-03-13 12:02:38,614] [ HABApp.file.events] DEBUG \| FileCreatedEvent(src_path='/habapp/config/rules/presence/.leave_arrival.py.1upIJ8', dest_path='', event_type='created', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.612 | [2024-03-13 12:02:38,612] [ HABApp.file.events] DEBUG \| FileOpenedEvent(src_path='/habapp/config/rules/presence/leave_arrival.py', dest_path='', event_type='opened', is_directory=False, is_synthetic=False) |
| | 2024-03-13 12:02:38.587 | [2024-03-13 12:02:38,585] [ HABApp.file.events] DEBUG \| FileOpenedEvent(src_path='/habapp/config/rules/presence/leave_arrival.py', dest_path='', event_type='opened', is_directory=False, is_synthetic=False)
Does the file get loaded again after the unload? If it's still there I would have expected that it gets unloaded and then immediately loaded again.
Nope, that's the full log. Habapp forgets about this file. What's interesting, when I copy files using scp, everythong works, rsync does it in some not compatible way. Another interesting thing: I tried to change behavior of rsync using --inplace flag, but it doesn't change anythong, habapp behaves in the same way. Ah, systems: rsync runs on OSX, copies files using ssh to Linux@arm (raspberry)
I can't make sense of it. Could you additionally show me the corresponding file events from the HABApp events?
They are on the event bus under HABApp.Files
No, the corresponding issued file events are missing. You typically find these in the HABApp_events.log under HABApp.Files
Ahh, I see. Unfortunately I have logging level set to WARN for events, and all is sent to Loki - as I mentioned, whole log is in the first post
You can set the logging level for the event file to INFO, then do rsync and then set it back to warning again. It will be picked up on the fly.