Farm-Data-Relay-System
Farm-Data-Relay-System copied to clipboard
timmbogner
Configuration System
Original discussion was held on PR https://github.com/timmbogner/Farm-Data-Relay-System/pull/14. Moving it here so it does not get lost.
For my use-case, having configuration values set up at compile time is quite inconvenient and rather impractical. This portable class based library approach would suit me much better. Ideally, being able to set up the Unit mac, gateway mac and reading ID's at runtime would be much better and easier to deploy. Perhaps using an overloaded fdrs.begin() or separate functions for this purpose. This way, on bootup, the program can retrieve the appropriate settings from config files in SPIFFS (for example). These settings can potentially be user configured (via webserver or other UI). I think there would also be value in doing something similar with the gateways. The trick would be to not sacrifice too much of the modularity / flexibility. I was thinking of defining a gateway type based on the port type to which it transmits (LoRa, ESPNow, Serial, etc.) Perhaps one can then consider creating multiple instances of such a gateway object, where each instance listens on a predefined port. This way, one can still implement a gateway in a single physical device that can handle multiple sensor types. If this makes any sense? Perhaps there is a better way of doing this? Also, for the gateways (MQTT specifically), to have WiFi credentials hard-coded is not ideal and would be hard to deploy. I would prefer to set this up using WiFiManager or something similar. Again, having the appropriate gateway mac's configurable at run-time would make it much easier to deploy.
@timmbogner
@lekrom Thanks! I definitely see your point on compile-time vs run-time. Way back in my notes I wrote "FDRS: Field Configurable" which was to be a method of using DIP switches or jumpers to configure devices. I like the idea of keeping config data in (the filesystem formerly known as) SPIFFS. The question is how to build a lightweight method of setting/changing it. Can you think of a good way to add webserver functionality without adding much battery time at startup? I've been planning a method for sensors to send special ESP-NOW broadcast packets and receive responses about nearby gateways. These packets will be set apart from DataReading packets by being smaller than them. Perhaps this could be employed to make a remote programming device that assigns config data via ESP-NOW? Serial? Maybe just a command to send it into webserver mode? I think I'm also seeing your concept with gateways. I'll be able to to discuss it a little better once I research classes and objects this week. Thanks again to both of you! I'll be in touch!
Glad to assist. I came across a tecnique on the 8266 where a double reset triggers a simple webserver. The device checks on bootup if the reset button was recently pressed twice. This is used on the iSpindel device ( an esp based tilting hydrometer uses to monitor beer fermentarion. https://github.com/universam1/iSpindel Could maybe do something like that. Also, it uses WiFiManager, a library that handles wifi login and credential storage for you. The library can be used to store additional config values. This is often used to store mqqt stuff and sonetimes blynk tokens for example. Should be useful here for gateway macs and the like. Could also maybe trigger "config mode" by sending a command msg.
The WiFiManager library has some nice examples for custom setting fields. It also shows how to trigger the config portal "on demand" using a pushbutton. https://github.com/tzapu/WiFiManager
@timmbogner Mind adding a label to this as a enhancement.
I would suggest doing the configuration via UART. This will have the advantage of not locking the platform in a single line of MCUs. This becomes relevant when other transport layers get implemented like ethernet, nRF24L01, 4G LTE, and E5 LoRa. Not everything has Wifi but just about everything as a serial port.
The on the fly gateway mac and reading ID configuration can be done with a set/get pair of function. Just about any no-volatile storage can be used to store the config.
That's a really good point, UART should always be kept as an option. Configurability via WiFi would be a big asset, though. I think that rather than depending on WiFi Manager I'd prefer to make a lighter version: just an AP with a captive portal that gives you a page with options, and then sets them using our set/get functions. I'm not sure if the non-ESP Arduino WiFi boards have AP capability but if so, the functionality could be extended to them too. @Devilbinder do you think triggering it with double-reset would be portable enough Being honest, It will be rare that I'm not using an ESP, so I don't mind adding an extra feature for it. I just don't want to break other compatibilities. I don't even have a LoRa board without an ESP attached, so I have never tested AVR compatibility. I just know it compiles. Edit: On second thought, maybe leaving it as just a UART interface and creating a separate programmer device would be best for now. Later on we can get fancier with it. I might use this as a trial for my new, smaller "network packet" format.
Having a way of changing the config parameters via a function provided by the library, should be flexible enough for the purpose of the library. We could play with different implementations in the examples. That way, users can implement their own way of dealing with the config based on their use-case and hardware choice. Doing too many fancy stuff in the library will undoubtedly add a lot of bloat. I will try to setup a few tests and start looking at some examples to this effect. I should hopefully get to it this weekend. I do not have a front end (Node-red) setup yet but I do have a bunch of ESP32 boards (with LoRa modules on) and some D1 Mini's lying around.
Gotcha, that makes sense. The new library is working now in the "dev" branch. Node Red is fairly easy to set up, but if you want to test a system without it: watch the serial monitor on the UART gateway that is feeding the MQTT gateway (#4 in the examples) and you should see all of the DataReadings being sent in JSON format. I would swear that my floats didn't used to add precision the way they do now (21.0345 becomes 21.03450012) but I've come to understand that this is just why floats aren't ideal.
Thanks. I will see if I can get a node-red instance fired up. I want to learn it any case so a good opportunity there. In the meanwhile, I was also thinking of testing with the serial monitor on a gateway. Should be easy enough...
@timmbogner I have somewhat given up on the idea of not locking the project into the esp. I like having RTOS available if I need it. Already have seen some stuff with the serial gateway that will cause issues. That will be simple to solve by using a RTOS thread (task).
On second though only the Gateway's need to be locked into ESP platform. The sensor can be whatever.
@Devilbinder This has been my outlook for a while, though for a few smaller reasons. In the sensor sketch I only had to make ESP-NOW optional and suddenly it was AVR compatible (I actually have to fix this a little bit... noticed a bug with spi yesterday). The gateway on the other hand needed a lot more to make it hardware independent, so I never tried. It seems like an acceptable limitation for the time being. The avr dudes will just have to chill 😎
I agree. A "no infrastructure wireless network" without wireless capability does not make much sense.