Docker/K8s implementation

[mddeff]

Hey @liamcottle, first I wanted to say, awesome project! Was able to get it and running very quickly.

I'm working on a Docker compose implantation right now and had a few quick questions:

1. Do mqtt.js and index.js need to interact with each other in any way? or is it all just mqtt.js writing to DB, and index.js displaying results? (trying to determine if they can be run as separate containers, and if so, any ports that need to be opened between them)

2. Is there any interest in a PR to this repo for a Docker image built off of GitHub Actions and associated docs/docker-compose.yml?

3. Same as 2, but a Kubernetes Helm chart

I understand the implications of merging 2 and 3 into this repo; it becomes 'official supported'. If thats something you're not ready to take on (I'm happy to help on an as-available), totally understand.

Either way, thanks for the awesome contribution to the Meshtastic ecosystem!

[mddeff]

Working branch

[liamcottle]

Hey @mddeff thanks for the feedback and please forgive my late reply, have been a bit busy, and wanted to put more than 5 seconds into the reply...

First off, to answer your questions;

Do mqtt.js and index.js need to interact with each other in any way? or is it all just mqtt.js writing to DB, and index.js displaying results? (trying to determine if they can be run as separate containers, and if so, any ports that need to be opened between them)

or is it all just mqtt.js writing to DB, and index.js displaying results?

Yep! They don't talk to each other directly at all, mqtt.js just collects packets it hears on MQTT and then stores them in the database, and index.js just runs the web api for the UI to pull data from the database.

So as long as both scripts have access to a shared database, it should work as expected, regardless of if you run them in different containers.

Is there any interest in a PR to this repo for a Docker image built off of GitHub Actions and associated docs/docker-compose.yml?

I didn't plan to add a Dockerfile or build a docker image, since it was originally such a simple project, but I will likely implement an actual build process with webpack etc in the future to build the .js and .css files etc. As it currently stands, when self hosting in an offline environment, some externally loaded scripts will not work. Such as vuejs, tailwind and leaflet.

Same as 2, but a Kubernetes Helm chart

I probably wouldn't be interested in adding this. I run k8s in prod for some bigger projects, and think it's a bit more complex for something like this project. If someone wants to run it in k8s, they can write their own config.

I understand the implications of merging 2 and 3 into this repo; it becomes 'official supported'.

Yeah I feel this. I will be keen on Docker support later on. Maybe we can get something merged in soon, but I do plan to change the structure a bit, so it would become stale pretty quick and need updating.

I did see in your branch, you were running the mqtt.js as a background process with &. It's been a hot minute since I wrote a docker file, but the mqtt.js script could crash at anypoint, so we would want to make sure this is an auto-restartable container etc.

Right now, I run both scripts as their own systemd service on my linux vm for my hosted map, and have had mqtt.js service exit and restart several times already.

Either way, thanks for the awesome contribution to the Meshtastic ecosystem!

Thanks! Happy to contribute however I can :) Still waiting on rak kits to arrive to get some solar nodes up and running! Those esp32 boards are too hungry...

[mddeff]

No worries on the delay, it's almost like we all have our own lives and I appreciate the thorough feedback!

I did see in your branch, you were running the mqtt.js as a background process with &.

Yep, that was a quick/dirty/ugly solution just for proof of concept. Since you've confirmed that index.js and mqtt.js don't need to talk to each other in any way; definitely going to break those out into separate containers for a docker-compose setup and let docker do its thing for process management/restarting.

Yeah I feel this. I will be keen on Docker support later on. Maybe we can get something merged in soon, but I do plan to change the structure a bit, so it would become stale pretty quick and need updating.

Completely understood; I'll probably just keep chugging along in my own repo for builds. Whenever you think the project is ready for images let me know and I'll happily help out.

I run k8s in prod for some bigger projects, and think it's a bit more complex for something like this project. If someone wants to run it in k8s, they can write their own config.

Honestly that's something that I'll likely maintain for myself anyways; I also do a lot of k8s/helm at work and its just easier for my brain in my home lab than going "Which host am I running that docker compose on again?" Also good exercise for building helm charts from scratch.

If you think its going to be a bit before adding docker support, feel free to close this issue and we can sync up whenever you think the project is ready. Thanks again!

[BillyBag2]

These two project are great! I have submodule version of the docker solution which should require less maintenance but requires understanding submodules. It should have a longer shelf life.

https://github.com/BillyBag2/meshtastic-map-docker

[mddeff]

Haha beat me to the punch @BillyBag2, I'm planning on refactoring the docker file this weekend to do everything in separate containers as mentioned above. Will definitely take a look at what you have!

[bbbenji]

I've also gotten this project running in Docker on my fork: https://github.com/bbbenji/meshmap-map

[BillyBag2]

@mddeff I used submodules so it's a separate, decoupled, repository. There is no need to keep the branches synced. Some people like sub modules, others do not. You could do the same with just a simple checkout script. I split the setup into three bits, migration, mqtt, and the map. They are all in the image but I intend to use the multi stage build, or docker compose, to make flavoured containers.Thats as far as I got. I want to do some map development at some point so may add that. I think map only den could join the default deployment and benefit from all the saved data.Have not really thought though sql development but perhaps the networks/volumes need to be a parameter so I can have a development network with different sql data but keep the default one collecting 24/7.

[liamcottle]

Basic docker compose support has been added in #44 closing this out for now :) Thanks!