docker
docker copied to clipboard
matomo-org
Official Helm chart
I plan to work on this, this week or next week.
Anybody has a succesful deployment on k8s? If you could share some experience, would be nice. It would be nice to collaborate on that!
@pierreozoux I do have it. I cant publish the files as they belong to the company I work for. But I can help you with a setup.
Can't you ask your company to release it open source?
You got the Dockerfile for free :)
I know how to do it, I already did Rocket.Chat, Mautic, Jitsi. I think it is nice to collaborate :)
But at the end, it is up to your company, indeed.
Tell them that the work to maintain it will be spread across various people and at the end of the day, it will be less expensive for your company. + it will give some datalove to your company, they can then say that they do open source. It is good to recruit devs/ops :)
I work for the Goverment it will take like 6 months to get an approval if so. But I can take a look on the data I have and see what can I release. Do you have already a kubernetes cluster? The installation was quite straightforward, so maybe I can help you with the service, ingress and networkpolice? Is that what you need?
The installation on k8s is indeed quite straight forward, just some issues with the latest 3.0.3 update. I'm working on it and will report back here.
Here is my running config, unfortunately there are two little issues with it:
- piwik config file is hardcoded with usernames and passwords for mail host and database. This should be done with an initContainer which generates the piwik config file from the database Secret and configMap. Just hadn't had time to do it.
- Since I need more memory for PHP I overwrite the shipped php.ini and restart the php worker processes in the fpm container. Not really a clean solution but WFM.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: piwik
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
name: piwik
labels:
app: piwik
role: frontend
spec:
# Unfortunately not in 1.5 yet
# This is here for future correction as soon as cluster is on 1.6 or above
# # These containers are run during pod initialization
# initContainers:
# - name: install
# image: busybox
# command:
# - sh
# - -c
# - cp /etc/config/config.ini.php /var/www/html/config/
# volumeMounts:
# - name: piwik-conf
# mountPath: "/etc/config/"
containers:
- image: piwik:3.0.3-fpm
name: piwik-php
lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- cp /etc/config/config.ini.php /var/www/html/config/; cp /etc/config/php.ini /usr/local/etc/php/php.ini; kill -USR2 1
ports:
- containerPort: 9000
name: piwik-port
volumeMounts:
- name: html-dir
mountPath: /var/www/html
- name: piwik-conf
mountPath: /etc/config/
- image: nginx:1.11.12
name: piwik-proxy
args: ["nginx", "-g", "daemon off;", "-c", "/etc/config/nginx.conf"]
ports:
- containerPort: 80
name: http
volumeMounts:
- name: piwik-nginx-conf
mountPath: /etc/config/
- name: html-dir
mountPath: /var/www/html
volumes:
- name: piwik-nginx-conf
configMap:
name: piwik-nginx-conf
- name: piwik-conf
configMap:
name: piwik-conf
- name: html-dir
emptyDir: {}
---
kind: ConfigMap
apiVersion: v1
metadata:
name: piwik-nginx-conf
data:
nginx.conf: |-
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
upstream backend {
server localhost:9000;
}
include /etc/nginx/mime.types;
default_type application/octet-stream;
gzip on;
gzip_disable "msie6";
server {
listen 80;
root /var/www/html/;
index index.php index.html index.htm;
location / {
try_files $uri $uri/ =404;
}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
location = /favicon.ico {
log_not_found off;
access_log off;
}
location ~ \.php$ {
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_intercept_errors on;
fastcgi_pass backend;
}
}
}
---
kind: ConfigMap
apiVersion: v1
metadata:
name: piwik-conf
data:
php.ini: |-
always_populate_raw_post_data=-1
extension=geoip.so
geoip.custom_directory=/var/www/html/misc
memory_limit=256M
config.ini.php: |-
; <?php exit; ?> DO NOT REMOVE THIS LINE
; file automatically generated or modified by Piwik; you can manually override the default values in global.ini.php by redefining them in this file.
[database]
host = "backend-sql"
username = "***"
password = "***"
dbname = "***"
tables_prefix = "piwik_"
[General]
proxy_client_headers[] = "HTTP_X_FORWARDED_FOR"
proxy_host_headers[] = "HTTP_X_FORWARDED_HOST"
salt = "***"
trusted_hosts[] = "***"
session_save_handler = dbtable
minimum_memory_limit = -1
[mail]
transport = "smtp"
port = "25"
host = "***"
type = "Plain"
username = "***"
password = "***"
encryption = "tls"
[Plugins]
Plugins[] = "CorePluginsAdmin"
Plugins[] = "CoreAdminHome"
Plugins[] = "CoreHome"
Plugins[] = "WebsiteMeasurable"
Plugins[] = "Diagnostics"
Plugins[] = "CoreVisualizations"
Plugins[] = "Proxy"
Plugins[] = "API"
Plugins[] = "ExamplePlugin"
Plugins[] = "Widgetize"
Plugins[] = "Transitions"
Plugins[] = "LanguagesManager"
Plugins[] = "Actions"
Plugins[] = "Dashboard"
Plugins[] = "MultiSites"
Plugins[] = "Referrers"
Plugins[] = "UserLanguage"
Plugins[] = "DevicesDetection"
Plugins[] = "Goals"
Plugins[] = "Ecommerce"
Plugins[] = "SEO"
Plugins[] = "Events"
Plugins[] = "UserCountry"
Plugins[] = "VisitsSummary"
Plugins[] = "VisitFrequency"
Plugins[] = "VisitTime"
Plugins[] = "VisitorInterest"
Plugins[] = "ExampleAPI"
Plugins[] = "RssWidget"
Plugins[] = "Feedback"
Plugins[] = "Monolog"
Plugins[] = "Login"
Plugins[] = "UsersManager"
Plugins[] = "SitesManager"
Plugins[] = "Installation"
Plugins[] = "CoreUpdater"
Plugins[] = "CoreConsole"
Plugins[] = "ScheduledReports"
Plugins[] = "UserCountryMap"
Plugins[] = "Live"
Plugins[] = "CustomVariables"
Plugins[] = "PrivacyManager"
Plugins[] = "ImageGraph"
Plugins[] = "Annotations"
Plugins[] = "MobileMessaging"
Plugins[] = "Overlay"
Plugins[] = "SegmentEditor"
Plugins[] = "Insights"
Plugins[] = "Morpheus"
Plugins[] = "Contents"
Plugins[] = "BulkTracking"
Plugins[] = "Resolution"
Plugins[] = "DevicePlugins"
Plugins[] = "Heartbeat"
Plugins[] = "Intl"
Plugins[] = "Marketplace"
Plugins[] = "ProfessionalServices"
Plugins[] = "UserId"
Plugins[] = "CustomPiwikJs"
Plugins[] = "Provider"
[PluginsInstalled]
PluginsInstalled[] = "Diagnostics"
PluginsInstalled[] = "Login"
PluginsInstalled[] = "CoreAdminHome"
PluginsInstalled[] = "UsersManager"
PluginsInstalled[] = "SitesManager"
PluginsInstalled[] = "Installation"
PluginsInstalled[] = "Monolog"
PluginsInstalled[] = "Intl"
PluginsInstalled[] = "CorePluginsAdmin"
PluginsInstalled[] = "CoreHome"
PluginsInstalled[] = "WebsiteMeasurable"
PluginsInstalled[] = "CoreVisualizations"
PluginsInstalled[] = "Proxy"
PluginsInstalled[] = "API"
PluginsInstalled[] = "ExamplePlugin"
PluginsInstalled[] = "Widgetize"
PluginsInstalled[] = "Transitions"
PluginsInstalled[] = "LanguagesManager"
PluginsInstalled[] = "Actions"
PluginsInstalled[] = "Dashboard"
PluginsInstalled[] = "MultiSites"
PluginsInstalled[] = "Referrers"
PluginsInstalled[] = "UserLanguage"
PluginsInstalled[] = "DevicesDetection"
PluginsInstalled[] = "Goals"
PluginsInstalled[] = "Ecommerce"
PluginsInstalled[] = "SEO"
PluginsInstalled[] = "Events"
PluginsInstalled[] = "UserCountry"
PluginsInstalled[] = "VisitsSummary"
PluginsInstalled[] = "VisitFrequency"
PluginsInstalled[] = "VisitTime"
PluginsInstalled[] = "VisitorInterest"
PluginsInstalled[] = "ExampleAPI"
PluginsInstalled[] = "RssWidget"
PluginsInstalled[] = "Feedback"
PluginsInstalled[] = "CoreUpdater"
PluginsInstalled[] = "CoreConsole"
PluginsInstalled[] = "ScheduledReports"
PluginsInstalled[] = "UserCountryMap"
PluginsInstalled[] = "Live"
PluginsInstalled[] = "CustomVariables"
PluginsInstalled[] = "PrivacyManager"
PluginsInstalled[] = "ImageGraph"
PluginsInstalled[] = "Annotations"
PluginsInstalled[] = "MobileMessaging"
PluginsInstalled[] = "Overlay"
PluginsInstalled[] = "SegmentEditor"
PluginsInstalled[] = "Insights"
PluginsInstalled[] = "Morpheus"
PluginsInstalled[] = "Contents"
PluginsInstalled[] = "BulkTracking"
PluginsInstalled[] = "Resolution"
PluginsInstalled[] = "DevicePlugins"
PluginsInstalled[] = "Heartbeat"
PluginsInstalled[] = "Marketplace"
PluginsInstalled[] = "ProfessionalServices"
PluginsInstalled[] = "UserId"
PluginsInstalled[] = "CustomPiwikJs"
PluginsInstalled[] = "Provider"
---
@maikotz assuming we are discussing a solution on the context of 11 principles the file cannot be hard-coded this is exactly what we are trying to avoid. @pierreozoux I might have missed the discussion but we seek to provide the configuration data as parameters to the deploy right?
@pierreozoux here goes, ive removed some sensitive data.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: piwiki
spec:
replicas: 1
template:
metadata:
labels:
app: piwiki
spec:
containers:
- name: piwiki
image: @TODO: REMOVED
imagePullPolicy: Always
ports:
- containerPort: 80
resources:
limits:
memory: 2Gi
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/rewrite-target: /
ingress.kubernetes.io/whitelist-source-range: 10.0.0.0/8,127.0.0.1/32
creationTimestamp: 2017-03-13T18:37:30Z
generation: 2
name: piwik
namespace: @TODO: ADD HERE
resourceVersion: "19307725"
spec:
rules:
- host: @TODO: ADD HERE
http:
paths:
- backend:
serviceName: piwik-1774830804-zmx1s
servicePort: 80
path: /
tls:
- hosts:
- @TODO: ADD HERE
status:
loadBalancer: {}
kind: NetworkPolicy
apiVersion: extensions/v1beta1
metadata:
name: piwiki
spec:
podSelector:
matchLabels:
app: piwiki
ingress:
- ports:
- protocol: TCP
port: 80
- protocol: TCP
port: 443
So what I would expect is that this section from config.ini.php to be configured through ENV vars:
[database] host = "@HOSTNAME" username = "@USERNAME" password = "@PASSWORD" dbname = "DNAME" tables_prefix = "blah_" charset = "utf8"
So I could invoke the container like:
kubectl run --HOSTANAME="blah" --USERNAME="blah" --PASSWORD="blah" --DBNAME="blah"-i -t piwiki.image:latest /bin/bash
@vfbsilva you're right it shouldn't be hardcoded, especially with the plugin list and everything else but it seems that piwik itself doesn't support it so far, there are other open issues and forum posts regarding the issue with hardcoded configs.
My suggestion would be an init-container that reads the environment variables and creates the config on-demand by replacing placeholders in the config.sample file
@maikotz I'm sorta new to the community and I do not want to start a flamme war so if you think it is better we can discuss it via mail. As long it is oki and as I find there is a misunderstanding regarding our expectations I will explain my view here. TL;DR; It does not support.
Problem, Discussion, Expectation Problem: Ethereal file systems: the data is created and destroyed within the app space (pod, container whatever), I need the deploy to be resilient. Ie: I've crashed a frontend, I've added another one, I want to be aple to give the database config as paramethers, hence I can have 1 job which makes all my deploys from a auth list for example.
Discussion: I don't know how an init container would work. But I ask, if I have multiple piwiks in a single namespace will this init container work for this kind of setup?
Culprit database section from config file: [database] host = "@HOSTNAME" username = "@USERNAME" password = "@PASSWORD" dbname = "DNAME" tables_prefix = "blah_" charset = "utf8"
Possible Solution(Expected behaviour, assuming Kubernets):
The database config belongs to a pod. Not to a service or deployment, it belongs to the pod itself. How do I invoke a pod:
kubectl run -i -t piwiki.image:latest /bin/bash
How could I possible provide the info to the pod?
kubectl run --HOSTANAME="blah" --USERNAME="blah" --PASSWORD="blah" --DBNAME="blah"-i -t piwiki.image:latest /bin/bash
How are enviroment variables supported on Kubernets: https://kubernetes.io/docs/tasks/configure-pod-container/define-environment-variable-container/
What I would expect to generate with my command:
apiVersion: v1
kind: Pod
metadata:
name: envar-demo
labels:
purpose: demonstrate-envars
spec:
containers:
- name: envar-demo-container
image: gcr.io/google-samples/node-hello:1.0
env:
- name: HOSTNAME
value: "blah"
- name: DBNAME
value: "blah"
- name: PASSWORD
value: "blah"
- name: USERNAME
value: "blah"
So no need of an init container IMHO. You could have a "sed/awk" script which replaces the config within the file but this is a hack not something really supported and would break on update. but @pierreozoux needed those files to run the image on kubernets.
@maikotz further reference: https://12factor.net/config
Sorry, priority shifted.. anybody, feel free to PR the official helm chart :)
Any plans on moving the Helm chart you have hosted at https://github.com/matomo-org/docker/tree/master/.examples/helm to the official helm charts repository? At least to the incubator one? https://github.com/helm/charts
@pierreozoux Quick question about https://github.com/matomo-org/docker/blob/705284f6ead4a73b78fefe1e4f11a8d8ffb033f5/.examples/helm/templates/configmap.yaml#L11
(Disclaimer: I know nothing about kubernetes and co.)
Does the linked nginx config use nginx as a reverse proxy or a primary webserver? Because in the latter case, Matomo wouldn't be secure by default as the .htaccess files are not read and therefore e.g. tmp/ would be public)
Maybe this should be compared with https://github.com/matomo-org/matomo-nginx
@appinteractive Yeah.. I got it working locally and in a gcp cluster for my tests. I just forked the original helm/charts repo today and will include my changes there and will let you know here... After that, I'll just need to refactor the code a little bit, include a better readme file, etc.. before creating a pull request in their helm/charts repo. ;]
@appinteractive Sorry for the delay.. Just super busy at work these days. So, I got it working and pushed to our fork here: https://github.com/thinkresearch/charts/tree/MATOMO/incubator/matomo
IMPORTANT: Didn't have time to create the README.md yet, but plan to do so during the next couple of hours/days.
Will go over the code again and see if I can refactor it, change something for best practices, etc.. before sending a PR to the official helm charts repo.
Please, let me know if you have any thoughts.
I saw that bitnami has a docker image for matomo. They provide quite a few quality charts / images. Maybe worth a look: https://github.com/bitnami/bitnami-docker-matomo
Well, I just successfully built a basic helm chart that uses the bitnami docker image. I might share it if someone is interested.
It's for internal deployment for now. Any feedback is appreciated.
https://github.com/jptissot/matomo-chart
Hello Guys, I am facing the issue where whenever my pod restarts it looses DB config. I am using the helm charts from ".example" folder. I see a lot of discussion here for initializing the DB config, but I am not clear on the solution. Can someone please help me with config. For some reason, the pod keeps crashing and it stops tracking. And we dont notice it for a day or two and loose usage as it needs to be set up again. Still need to figure out why it keeps crashing. But at least for next time, we wont loose data.
@kalib so still not success?
Sorry @appinteractive I no longer use Matomo... :/ I am no longer working on that.
Hello Guys, I am facing the issue where whenever my pod restarts it looses DB config. I am using the helm charts from ".example" folder. I see a lot of discussion here for initializing the DB config, but I am not clear on the solution. Can someone please help me with config. For some reason, the pod keeps crashing and it stops tracking. And we dont notice it for a day or two and loose usage as it needs to be set up again. Still need to figure out why it keeps crashing. But at least for next time, we wont loose data.
Sounds and looks like the example Chart is missing a volume for the created configuration, i. E. the data is written inside the running container to some ephemeral location and then lost on restart.
The chart linked in the comment above is working for us in Production with some minor inconveniences (I.e missing cronjob and config from configmap would be nicer).
I'm also working on a helm chart: https://gitlab.com/ideaplexus/helm/matomo
Features:
- using matomo with apache
- using mariadb with replication
- support for traefik 2
- using configmap for config
Planned/Not working yet
- using redis for QueuedTracking
- config can only enabled in second deploy, since there is (currently) no way to create tables / default user / site by cli
- archive cronjob
- automatic download of maxmid geoip database with cronjob
PS: Documentation is coming, feedback would be grateful.
