Improve flexibility of configuration via env. vars

[sanikolaev]

trafficstars

Currently it's possible to configure searchd via env. vars, but it's limited by only those settings that are hardcoded in manticore.conf, so some new and undocumented settings are not working, e.g. -e searchd_query_log_commands=1. The task is to refactor the feature, so it's possible to pass arbitrary settings via env. vars.

[djklim87]

Done in https://github.com/manticoresoftware/docker/pull/56

[sanikolaev]

Reopening since the CI failed https://github.com/manticoresoftware/docker/actions/runs/6947961151/job/18902901514

[djklim87]

I just rerun job and all fine now. It happens sometimes. Maybe because cronjob complete at the same second as container runs

[sanikolaev]

It's not working for me:

snikolaev@dev2:~$ docker run  -e searchd_listen='9307:mysql' -e EXTRA=1 --name manticore --rm -d manticoresearch/manticore:dev && echo "Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time" && until docker logs manticore 2>&1 | grep -q "accepting connections"; do sleep 1; echo -n .; done && echo started && docker exec -it manticore bash -c 'cat /etc/manticoresearch/manticore.conf.debug' && docker stop manticore
96cd544291948eb0abf031f5b18e8c84d626f792e819957079d15b864f701566
Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time
...started
searchd {
    listen = 9306:mysql41
    listen = /var/run/mysqld/mysqld.sock:mysql41
    listen = 9308:http
    listen = 172.17.0.12:9312
    listen = 172.17.0.12:9315-9325:replication
    log = /var/log/manticore/searchd.log
    query_log = /var/log/manticore/query.log
    pid_file = /var/run/manticore/searchd.pid
    data_dir = /var/lib/manticore
}
manticore

[djklim87]

Done in https://github.com/manticoresoftware/docker/commit/5d426e5ea4c54a13ee78aafd1a374554a48821c2

[tomatolog]

seems this fix is cause of the inconsistency in the env option of the docker \ docker-compose file that cause the issue https://github.com/manticoresoftware/manticoresearch/issues/2500

ie the docker-compose.yaml that allows to issue CLUSTER CREATE well with 6.2.12 release

    environment:
      - EXTRA=1
      - searchd_listen=9306:mysql41
    cap_add:
      - IPC_LOCK

in the 6.3.6 release fails with the error message

mysql> create cluster xxx;
ERROR 1064 (42000): cluster 'xxx' is not ready, starting

that is due to searchd_listen now overrides all default port settings and daemon starts as

starting daemon version '6.3.6 593045790@24080214 (columnar 2.3.0 88a01c3@24052206) (secondary 2.3.0 88a01c3@24052206) (knn 2.3.0 88a01c3@24052206)' ...
listening on all interfaces for mysql, port=9307
prereading 0 tables

however at the 6.2.12 release searchd_listen adds new port mappings to the default port settings and daemon starts as

listening on all interfaces for mysql, port=9307
listening on all interfaces for mysql, port=9306
listening on UNIX socket /var/run/mysqld/mysqld.sock
listening on 10.201.2.2:9312 for sphinx and http(s)
listening on all interfaces for sphinx and http(s), port=9308

not sure is this a bug that should be fixed or planned change that just lack of the description in the changelog as now the changelog looks like :exclamation:Issue #47 Improved flexibility of configuration via environment vars.

[sanikolaev]

not sure is this a bug that should be fixed or planned change that just lack of the description in the changelog as now the changelog looks like

The readme was improved in https://github.com/manticoresoftware/docker/commit/3ebe4bffeab6832c4de412899e8c3b5c500114a7

Closing this issue.