Package differently?

[Jeppedy]

I thank you so much for writing this and putting it together. I wonder if now that you have it rolling along, you might consider publishing out an image. We could just pull down the image, create our volumes and drop the env file(s) into the external volume. The way it is now, I build everything by following your directions, but then I make a Docker Compose inside Portainer that references the previously-built images.

Could be so much more convenient if you published the image. But I know, only so much time in the day!

[plaintextpackets]

Thanks for posting yeah I'm considering that, just need the time to compile it all into one image, right now using separate images made it super quick to get it up and running

[plaintextpackets]

I think I might get the main image compiled then at least people don't have to build it from scratch. The others are just common images like Redis and Grafana

[plaintextpackets]

I now realize the other issue, I didn't want to pay for Docker hosting LOL

[Jeppedy]

I didn't realize you had to. I'll look into it, but I thought the docker.io hub was free to post to.

On Sat, Apr 27, 2024, 10:27 plaintextpackets @.***> wrote:

I now realize the other issue, I didn't want to pay for Docker hosting LOL

— Reply to this email directly, view it on GitHub https://github.com/plaintextpackets/netprobe_lite/issues/13#issuecomment-2080904384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB32XY7CL32DGUA5CY7FKNDY7O7XLAVCNFSM6AAAAABG3ZO3FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBQHEYDIMZYGQ . You are receiving this because you authored the thread.Message ID: @.***>

[Jeppedy]

Looks like until it takes off, you could use the free plan. If you need more, then yeah, it's $60/year. True.

On Sat, Apr 27, 2024, 10:27 plaintextpackets @.***> wrote:

I now realize the other issue, I didn't want to pay for Docker hosting LOL

— Reply to this email directly, view it on GitHub https://github.com/plaintextpackets/netprobe_lite/issues/13#issuecomment-2080904384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB32XY7CL32DGUA5CY7FKNDY7O7XLAVCNFSM6AAAAABG3ZO3FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBQHEYDIMZYGQ . You are receiving this because you authored the thread.Message ID: @.***>

[plaintextpackets]

Yeah just read through it. Ok I'll add that to the backlog. I also want to test a smaller image

[Jeppedy]

If I can help you in any way, please let me know.

On Sat, Apr 27, 2024, 15:04 plaintextpackets @.***> wrote:

Yeah just read through it. Ok I'll add that to the backlog. I also want to test a smaller image

— Reply to this email directly, view it on GitHub https://github.com/plaintextpackets/netprobe_lite/issues/13#issuecomment-2081167575, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB32XY2ZY3H5LTEMAV7RDD3Y7QADRAVCNFSM6AAAAABG3ZO3FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGE3DONJXGU . You are receiving this because you authored the thread.Message ID: @.***>

[PISTOLCUPCAKES]

I'll second this request and add (although I'm not sure if it's a better/worse option in this instance) there's also the Github Container Registry.

[plaintextpackets]

I compiled the image and posted to Dockerhub, if you have cycles to help me test:

https://github.com/plaintextpackets/netprobe_lite/tree/compiled_image

It's a pretty simple change and should avoid any of the issues with Docker building on various platforms

[plaintextpackets]

https://github.com/plaintextpackets/netprobe_lite/releases/tag/v1.1.0

[plaintextpackets]

Done in v1.1.0

[PISTOLCUPCAKES]

Not sure if I'm missing something here or what but I can't get it to run using the dockerhub image. I've cloned the github repo and pulled down the docker image. I've tried passing path <github_repo_path>:/netprobe_lite and it still fails to start with no logs. Can you share your bind variables/paths for using the image? Something else I may be doing wrong here?

[Jeppedy]

Testing now... Away from home, and it seems something isn't quite right. I'll debug tomorrow. Likely on my end...

On Sun, Apr 28, 2024, 10:34 plaintextpackets @.***> wrote:

I compiled the image and posted to Dockerhub, if you have cycles to help me test:

https://github.com/plaintextpackets/netprobe_lite/tree/compiled_image

It's a pretty simple change and should avoid any of the issues with Docker building on various platforms

— Reply to this email directly, view it on GitHub https://github.com/plaintextpackets/netprobe_lite/issues/13#issuecomment-2081506010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB32XY6EK4XUGNWMPXVWSRDY7UCHLAVCNFSM6AAAAABG3ZO3FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGUYDMMBRGA . You are receiving this because you authored the thread.Message ID: @.***>

[plaintextpackets]

Hmm are you still starting up via compose? The only image that I uploaded was for the main Netprobe image but still needs redis prom grafana which I did not combine

[Jeppedy]

Got it!

Pulled out some of the unneeded entries to slim it down and avoid conflicts for some (especially the network sections)

[plaintextpackets]

Does it work now?

[Jeppedy]

NetProbe using shared repository

services: redis: restart: unless-stopped container_name: netprobe-redis image: "redis:latest" volumes: - /home/jherr/docker-containers/netprobe_lite/config/redis/redis.conf:/etc/redis/redis.conf

netprobe: restart: unless-stopped container_name: netprobe-probe image: "plaintextpackets/netprobe:latest" volumes: - /home/jherr/docker-containers/netprobe_lite:/netprobe_lite environment: MODULE: "NETPROBE"

presentation: restart: unless-stopped container_name: netprobe-presentation image: "plaintextpackets/netprobe:latest" volumes: - /home/jherr/docker-containers/netprobe_lite:/netprobe_lite environment: MODULE: "PRESENTATION"

prometheus: restart: unless-stopped container_name: netprobe-prometheus image: "prom/prometheus" volumes: - /home/jherr/docker-containers/netprobe_lite/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml

grafana: restart: unless-stopped image: grafana/grafana-enterprise container_name: netprobe-grafana volumes: - /home/jherr/docker-containers/netprobe_lite/config/grafana/datasources/automatic.yml:/etc/grafana/provisioning/datasources/automatic.yml - /home/jherr/docker-containers/netprobe_lite/config/grafana/dashboards/main.yml:/etc/grafana/provisioning/dashboards/main.yml - /home/jherr/docker-containers/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json ports: - '3001:3000' ``

[Jeppedy]

Does it work now?

Works great! See my compose, above. I had to name it docker-compose.yaml, BTW, to get it found by the default name sought by Docker Compose

[Jeppedy]

Wait... Runs great, but when I stop and restart the stack, Grafana resets back to the default password and runs the migration jobs again. Is there a volume that needs to be mapped to local? Or is it resetting a startup file in that location that you expect to be there from your container, maybe?

[plaintextpackets]

So I think I need to explicitly map some more volumes. On both my test machines (Windows and Ubuntu) when Docker runs it automatically persists the data, but maybe that is just a default behavior.

Going to test some things, any ideas appereciated

[Jeppedy]

How are you referencing the Prometheus data store from Grafana? Are you using the container name or a hard-coded IP? I'm guessing your doing it right or it likely wouldn't connect at all...

[plaintextpackets]

I think I know the issue. I am not explicitly providing a data storage volume for either prometheus or grafana. For some reason on my devices that data somehow gets stored in a temporary volume but then kept, whereas others it gets cleaned up.

I should be doing something like this:

prometheus: restart: always container_name: netprobe-prometheus image: "prom/prometheus" volumes: - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml - prometheus_data:/prometheus/data # Mount a volume for data persistence networks: - custom_network # Attach to the custom network

volumes: prometheus_data: # Define a volume for Prometheus data persistence

[Jeppedy]

Try adding this volume... it’s stored in the grafana database which can be sqlite3 database (default, stored in /var/lib/grafana/grafana.db), postgres or mysql (depending on how you configured grafana).

I'm concerned though because I'm not familiar with how to map both a specific file and a parent directory. Maybe we should let it run to create default files, then stop, copy in dashboards file and restart?

[Jeppedy]

Makes sense. Same issue with Grafana for user accounts. See previous post.

On Sun, Apr 28, 2024, 16:26 plaintextpackets @.***> wrote:

I think I know the issue. I am not explicitly providing a data storage volume for either prometheus or grafana. For some reason on my devices that data somehow gets stored in a temporary volume but then kept, whereas others it gets cleaned up.

I should be doing something like this:

prometheus: restart: always container_name: netprobe-prometheus image: "prom/prometheus" volumes:

• ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml - prometheus_data:/prometheus/data # Mount a volume for data persistence networks:
• custom_network # Attach to the custom network

volumes: prometheus_data: # Define a volume for Prometheus data persistence

— Reply to this email directly, view it on GitHub https://github.com/plaintextpackets/netprobe_lite/issues/13#issuecomment-2081643701, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB32XY5RUTW3OJASKVOBOXLY7VLOBAVCNFSM6AAAAABG3ZO3FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGY2DGNZQGE . You are receiving this because you modified the open/close state.Message ID: @.***>

[plaintextpackets]

I'll get something together this evening and post for testing

[Jeppedy]

Confirmed... /var/lib/Grafana needs to persist, or at least one of the files in it does. And mapping a vol to a file name only works as expected when the file exists before starting the first time, else docker makes a folder the name of the file.

[plaintextpackets]

Here's what I'm testing right now:

1. Created two folders in the repo: /data/grafana /data/prometheus

2. Set gitignores to ignore the contents which are created at runtime in the repo

3. Set the build config for those two as follows:

prometheus: restart: always container_name: netprobe-prometheus image: "prom/prometheus" volumes: - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml - ./data/prometheus:/prometheus command: - '--config.file=/etc/prometheus/prometheus.yml' - '--storage.tsdb.path=/prometheus' networks: - custom_network # Attach to the custom network

grafana: restart: always image: grafana/grafana-enterprise container_name: netprobe-grafana volumes: - ./config/grafana/datasources/automatic.yml:/etc/grafana/provisioning/datasources/automatic.yml - ./config/grafana/dashboards/main.yml:/etc/grafana/provisioning/dashboards/main.yml - ./config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json - ./data/grafana:/var/lib/grafana ports: - '3001:3000' networks: - custom_network # Attach to the custom network

---

This seems to work, but would be a huge help if you tested.

My thought is you create the empty folders for the data and map, but don't let any of the data files into the repo. That way its nice and clean and all data should persist in the same place.

[Jeppedy]

So copy no files manually... Just clone the Git repository and fire it up, yes?

I'll have to test tonight. All my testing today has been from my cell phone! That's more than I want to attempt on the small keyboard and screen! Lol

[plaintextpackets]

Yep that's exactly it! Still keeping it with no manual mapping or folder creation, those folders will come down with the repo

Thanks mate

[PISTOLCUPCAKES]

I was able to get this running by just cloning the git repo, editing .env for my DNS server, and running docker compose up. All looks good!