Modules not getting reloaded

[tomvanderlee]

Hi,

Me being a programmer I like to use AppDaemon for all my HA automations. I really like AppDeamon, but I have one issue.

My current apps folder structure is as follows:

• apps.yaml
• app1.py
• module1
  • __init__.py
  • generic.py

App1 uses generic code from module1/generic.py. When I am programming and I make a mistake in module1/generic.py it logs the error in the error log when app1.py is reloaded (which is what I expect). When I fix the error in the module1/generic.py and reload app1.py it still throws the same error. It seems to me the module code is cached so I remove the pycache folders and reload app1.py, but event then it still throws the error.

The only way for me to reload the module code is to restart AppDaemon, which is not a very nice solution in my opinion.

Maybe this is already known, but I can't find any documentation about this.

Thanks, Tom

Running AppDaemon 4.0.1.

[ReneTode]

@tomvanderlee there are a few ways to deal with the kind of problems you got.

first option: make generic.py a real app. add yaml for it and then you can use the functions in that file with the get_app() function second option: make generic.py a real app and use dependies to make app 1 depend on generic.py third option: make use from the depends_on_module() function.

all these options make that if you change something in generic, app1 will be reloaded. i dont know which option you are using, but at this point there is somewhere a small bug which makes that when an app isnt terminated well the app runs double.

you can see that in the admin interface, the callbacks generated by the app are there twice. but in most cases if you make use from 1 of these 3 options everything goes right.

edit: and remember, if you want to make use from AD in optimal ways then its wise to rethink the ways you were used to programming in python. because of the reloading and always running structure from AD, there are other structure types better then what you were used too.

AD is an appbased system. and an app is the py class + its acompaniying YAML.

[matejdro]

I've tried option 3 with AppDaemon 4.0.3 and it does not appear to work. Dependent module will load once and then never reload, even after changing stuff in dependent module.

This is what I did:

dependent_module.py:

TEST_STRING = "ABC"

app.py:

from dependent_module import TEST_STRING

class TestApp(hass.Hass):
    async def initialize(self):
        await self.depends_on_module("dependent_module")

        self.log(f"TEST: {TEST_STRING}")

When I run this for the first time, it prints ABC like it should. But after I change the TEST_STRING inside dependent_module.py, appdaemon log does print Reloading Module: /conf/apps/dependent_module.py, but initialize method of the TestApp still prints ABC.

I've also tried adding this to the app yaml it wasn't any better:

global_modules:
  - dependent_module

app:
  module: app
  class: TestApp
  global_dependencies:
    - dependent_module

edit: and remember, if you want to make use from AD in optimal ways then its wise to rethink the ways you were used to programming in python. because of the reloading and always running structure from AD, there are other structure types better then what you were used too.

So what is the recommended way to create utility constants/classes/functions that are shared between multiple apps? Should I make Utils app that contains all of those? It seems a bit wasteful to run whole another app that does essentially nothing.

[ReneTode]

so you think its wastefull to create another app for general stuff, but it isnt wastefull to create a general lib?

i dont know if it could be related to the fact that you use async in your apps. AD is build to have threaded sync apps and only recently the possibility to use async is added. still the advised way is to use sync, unless you cant. and if you use async its not supported.

depends_on_module is added around the same time. get_app is inside AD for over 5 years and absolutely working as expected.

and to make sure you didnt run into something that is already repaired please update to 4.0.5

[matejdro]

Ok, wasteful was not the right word. My beef with the app approach is mostly this:

Instead of simply calling

some_util_method()

I must call

utils = self.get_app("utils")
utils.some_util_method()

i dont know if it could be related to the fact that you use async in your apps.

I've tried with sync app and problem is still there:

from appdaemon.plugins.hass.hassapi import Hass

from dependent_module import TEST_STRING


class TestApp(Hass):
    def initialize(self):
        self.depends_on_module("dependent_module")

        self.log(f"TEST: {TEST_STRING}")

and to make sure you didnt run into something that is already repaired please update to 4.0.5

Sorry, I thought I was already on the latest version, but it looks like I was hit with #1026. But unfortunately, 4.0.5 does not fix things.

[ReneTode]

you leave out a bit.

instead of

import some_module as mod
...
    self.depends_on_module("some_module")
   mod.some_function()

you use

utils = self.get_app("utils")
utils.some_function()

or

self.get_app("utils").some_function()

i also believe your class isnt correct, which makes that functions wont work. it should be:

import appdaemon.plugins.hass.hassapi as hass
class TestApp(hass.Hass):

and not

from appdaemon.plugins.hass.hassapi import Hass
class TestApp(Hass):

allthough thats probably another way of writing, but you import less (and maybe stuff that is needed not)

it can also be that you import just a part from the module you depend on. try it like the example in the docs.

>>> import somemodule
>>> import anothermodule
>>> # later
>>> self.depends_on_module([somemodule)

try it as string or as list of strings.

[matejdro]

Yeah, I can use get_app but my whole point was that utility functions should just be static functions, I shouldn't need an instance of some class to access them. That is why I think this is valid bug report, not just "you are using it wrong" type of situation.

Those different imports are just different ways to import same thing into python.

[ReneTode]

Those different imports are just different ways to import same thing into python.

there is an extrem difference between: import some_module and from some_module import something

and its possible that 1 thing works, and the other not.

maybe for 1 moment you should stop thinking about how you think it should be, and think about how you can adchieve what you want. the result is important, not the road to it.

[matejdro]

and its possible that 1 thing works, and the other not.

Sorry, I do not want to sound condescending, but If imports were broken, nothing would work. But apps work normally, just not the static imports.

maybe for 1 moment you should stop thinking about how you think it should be, and think about how you can adchieve what you want. the result is important, not the road to it.

I agree with you completely and I'm doing that. But the fact that there is a workaround does not make that bug report any less valid. It is still a bug in AppDaemon.

[ReneTode]

its only a bug if you try it exactly as the docs say and it doesnt work. if you try it at a different way, it cant be defined as bug ;)

i didnt check the code, but just for fun lets state this:

self.depends_on_module([somemodule) can expect that somemodule is imported, for it to reload it. but you do from somemodule import something. now there is nothing to reload, because somemodule is never imported. and even if it does reload that module, its imported different then you did, so it cant work.

the docs say:

import a
...
self.depends_on_module(a)
self.log(a.something)

so my suggestion is to see if that indeed doesnt work before you talk about a bug.

if the above works and

from a import b
... 
self.depends_on_module(a)
self.log(b)

doesnt you could add that as a feature request ;)

[matejdro]

Well, docs claim that AppDaemon runs on Python. And importing static variables/method via from keyword are part of python feature set.

Anyway, I've tested your method and it works! It looks like bug only applies when using from a import b. Thanks a lot for your help so far.

[ReneTode]

like i said its no bug.

depends on module re-import the module you give the name from. it cant read the code lines from the python script.

so its not possible to create a function (without jumping trough a lot of hoops and buggy solutions) that reimports a part from a module. same thing with import as ... that also cant be recreated without the user providing more info.

so a feature request could be to make it possible to do

self.depends_on_module({"import": "something", "from": "module", "as": "a"})

the docs tell you what can be done. you tried something that isnt in the docs. not everything that is possible in python automaticly applies to AD api.

[matejdro]

Should I create a feature separate ticket for feature request then?

[ReneTode]

you could, but for that you need to give the devs a good reason why they should take the extra efford.

i personally dont see the reason why they should add it. there is a working feature, and changing it would not make it better in my eyes. it would give the option to import in different styles, but personally i dont think its worth the efford, certainly not because they have little time IRL, and there is way more things they can do for AD.

you could also lookup how the function works and try to change it yourself, and when you got it working, then you could make a PR. that would give you more chance that it will be intergrated.

in the end its up to you.

[matejdro]

Alright, I totally understand. Thanks for your help.

[markusressel]

I don't want to stir up the pot again, but I want to mention that I wound up in the exact same rabbit hole as @matejdro, because I assumed that "standard" imports (taking into account my very limited understanding of how python works) would "just work" as they normally would.

I also tried to find a suggested way to do this in the docs but wasn't able to, maybe there is a possibility to enhance that and make it more clear that the opinionated way to access static functions in other files is via the mechanism provided by AppDaemon, and not via imports (even though that doesn't apply when using python libraries)? Maybe I didn't research good enogh, idk, at least I found this 3 year old thread :sweat_smile:

Thx for reading!

[markusressel]

Actually, I was able to somewhat work around this, just leaving this here in case it helps someone else:

Still kinda cumbersome, since the imports could probably also be figured out by AppDaemon automatically and you have to remember to create an app for everything yourself, but this seems to work and also allows reloading to work. Cheers!