citusdata
Locale issue with en_GB.UTF-8
Hi All,
I am trying to use an extended image of PostgreSQL with en_GB.utf8 locale as explained in the official docker documentation at https://hub.docker.com/r/_/postgres/
so my Dockerfile looks like
FROM postgres:10.3
RUN localedef -i en_GB -c -f UTF-8 -A /usr/share/locale/locale.alias en_GB.UTF-8
ENV LANG en_GB.utf8
ARG VERSION=7.4.1
LABEL maintainer="Citus Data https://citusdata.com" \
org.label-schema.name="Citus" \
org.label-schema.description="Scalable PostgreSQL for multi-tenant and real-time workloads" \
org.label-schema.url="https://www.citusdata.com" \
org.label-schema.vcs-url="https://github.com/citusdata/citus" \
org.label-schema.vendor="Citus Data, Inc." \
org.label-schema.version=${VERSION} \
org.label-schema.schema-version="1.0"
ENV CITUS_VERSION ${VERSION}.citus-1
# install Citus
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
&& curl -s https://install.citusdata.com/community/deb.sh | bash \
&& apt-get install -y postgresql-$PG_MAJOR-citus-7.4=$CITUS_VERSION \
postgresql-$PG_MAJOR-hll=2.10.2.citus-1 \
postgresql-$PG_MAJOR-topn=2.0.2 \
&& apt-get purge -y --auto-remove curl \
&& rm -rf /var/lib/apt/lists/*
# add citus to default PostgreSQL config
RUN echo "shared_preload_libraries='citus'" >> /usr/share/postgresql/postgresql.conf.sample
# add scripts to run after initdb
COPY 000-configure-stats.sh 001-create-citus-extension.sql /docker-entrypoint-initdb.d/
# add health check script
COPY pg_healthcheck /
HEALTHCHECK --interval=4s --start-period=6s CMD ./pg_healthcheck
The modified image gets generated without any issues however, when I try to start the cluster the master container remains in an unhealthy state and never reaches healthy state. Following is the log for the issue.
master_1 | The files belonging to this database system will be owned by user "postgres".
master_1 | This user must also own the server process.
master_1 |
master_1 | The database cluster will be initialized with locale "en_GB.utf8".
master_1 | The default database encoding has accordingly been set to "UTF8".
master_1 | The default text search configuration will be set to "english".
master_1 |
master_1 | Data page checksums are disabled.
master_1 |
master_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
master_1 | creating subdirectories ... ok
master_1 | selecting default max_connections ... 100
master_1 | selecting default shared_buffers ... 128MB
master_1 | selecting dynamic shared memory implementation ... posix
master_1 | creating configuration files ... ok
master_1 | running bootstrap script ... ok
master_1 | performing post-bootstrap initialization ... ok
master_1 |
master_1 | WARNING: enabling "trust" authentication for local connections
master_1 | You can change this by editing pg_hba.conf or using the option -A, or
master_1 | --auth-local and --auth-host, the next time you run initdb.
master_1 | syncing data to disk ... ok
master_1 |
master_1 | Success. You can now start the database server using:
master_1 |
master_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
master_1 |
master_1 | ****************************************************
master_1 | WARNING: No password has been set for the database.
master_1 | This will allow anyone with access to the
master_1 | Postgres port to access your database. In
master_1 | Docker's default configuration, this is
master_1 | effectively any other container on the same
master_1 | system.
master_1 |
master_1 | Use "-e POSTGRES_PASSWORD=password" to set
master_1 | it in "docker run".
master_1 | ****************************************************
master_1 | waiting for server to start....2018-07-13 09:06:44.614 UTC [89] LOG: number of prepared transactions has not been configured, overriding
master_1 | 2018-07-13 09:06:44.614 UTC [89] DETAIL: max_prepared_transactions is now set to 200
master_1 | 2018-07-13 09:06:44.659 UTC [89] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
master_1 | 2018-07-13 09:06:44.845 UTC [90] LOG: database system was shut down at 2018-07-13 09:06:17 UTC
master_1 | 2018-07-13 09:06:44.944 UTC [89] LOG: database system is ready to accept connections
master_1 | done
master_1 | server started
master_1 | ALTER ROLE
master_1 |
master_1 |
master_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/000-configure-stats.sh
master_1 |
master_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/001-create-citus-extension.sql
master_1 | BEGIN
master_1 | 2018-07-13 09:06:45.415 UTC [116] LOG: starting maintenance daemon on database 12995 user 10
master_1 | 2018-07-13 09:06:45.415 UTC [116] CONTEXT: Citus maintenance daemon for database 12995 user 10
master_1 | CREATE EXTENSION
master_1 | UPDATE 1
master_1 | COMMIT
master_1 |
master_1 |
**master_1 | 2018-07-13 09:06:47.293 UTC [89] LOG: received fast shutdown request
master_1 | waiting for server to shut down....2018-07-13 09:06:47.334 UTC [89] LOG: aborting any active transactions
master_1 | 2018-07-13 09:06:47.335 UTC [116] FATAL: terminating connection due to administrator command
master_1 | 2018-07-13 09:06:47.335 UTC [116] CONTEXT: Citus maintenance daemon for database 12995 user 10
master_1 | 2018-07-13 09:06:47.337 UTC [89] LOG: worker process: Citus Maintenance Daemon: 12995/10 (PID 116) exited with exit code 1
master_1 | 2018-07-13 09:06:47.339 UTC [89] LOG: worker process: logical replication launcher (PID 97) exited with exit code 1
master_1 | 2018-07-13 09:06:47.339 UTC [91] LOG: shutting down
master_1 | ...2018-07-13 09:06:50.597 UTC [89] LOG: database system is shut down
master_1 | done
master_1 | server stopped**
master_1 |
master_1 | PostgreSQL init process complete; ready for start up.
master_1 |
master_1 | 2018-07-13 09:06:50.668 UTC [1] LOG: number of prepared transactions has not been configured, overriding
master_1 | 2018-07-13 09:06:50.668 UTC [1] DETAIL: max_prepared_transactions is now set to 200
master_1 | 2018-07-13 09:06:50.668 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
master_1 | 2018-07-13 09:06:50.668 UTC [1] LOG: listening on IPv6 address "::", port 5432
master_1 | 2018-07-13 09:06:50.728 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
master_1 | 2018-07-13 09:06:50.835 UTC [125] LOG: database system was shut down at 2018-07-13 09:06:50 UTC
master_1 | 2018-07-13 09:06:50.879 UTC [1] LOG: database system is ready to accept connections
I am simply unable to understand why this is happening as it is just a change in the LANG / encoding. Could someone form the developer/maintainer / user community shed some light on this ?
Also, if somebody can guide me how to use Citus with en_GB.utf8 support that would really be a big help.
Thanks
devopxy
When I tried with a vanilla postgres docker container with en_GB.UTF-8 extension it works smoothly. The only change that you need to make is using the template 0 as stated in
https://stackoverflow.com/questions/18870775/how-to-change-the-template-database-collection-coding-on-postgresql]
However, when you try using en_GB.UTF-8 extension in a Citus cluster / master / workers with template 0 .. same issue as stated above !
I just attempted a quick reproduction of this using your Dockerfile, and I'm getting some inconsistent results. Could you specify the syntax you're using to build, launch, and connect?
-Colton
Hi @colton-citus ,
I am following following procedure >
1. Build >
docker build -t username/image:tag .
2. Launch >
Launching the cluster usingdocker-compose up -d replacing the image with the custom image <username/image:tag>
version: '2.1'
services:
master:
container_name: "${COMPOSE_PROJECT_NAME:-citus}_master"
image: '<username/image:tag> '
ports: ["${MASTER_EXTERNAL_PORT:-5432}:5432"]
labels: ['com.citusdata.role=Master']
worker:
image: '<username/image:tag> '
labels: ['com.citusdata.role=Worker']
depends_on: { manager: { condition: service_healthy } }
manager:
container_name: "${COMPOSE_PROJECT_NAME:-citus}_manager"
image: 'citusdata/membership-manager:0.2.0'
volumes: ['/var/run/docker.sock:/var/run/docker.sock']
depends_on: { master: { condition: service_healthy } }
Connecting
Connecting the cluster using pgAdmin 4
We have tried the setup on Debain 9 and Centos 7 on both the systems we could get the vanilla postgres container to create databases with en_GB.utf8 encoding but with Citus extension could not create. Following is the error message you get >
ERROR: invalid locale name: "en_GB.UTF-8"
Thanks !
Thanks for the clarification! It sounds like you're able to connect to the container to try creating a database, so I'm guessing we got past the initial trouble and it's now running- let me know if I'm badly mistaken. Feel free to ping me on our public Slack channel (I'm colton on there ) to hammer this out in closer to real-time if that works better for you.
With that docker-compose.yml, it worked fine for me. I've included output showing successful creation with en_GB.UTF-8 collation below so we can be sure if my testing is differing from yours somehow.
Perhaps there's a Docker version issue. I tried this on 18.03.1-ce - what's your Docker version?
It's possible that there's a problem with pgAdmin. Does connecting using docker exec as shown the snippet below have the same symptoms?
Perhaps this is only happening when you try CREATE DATABASE statements? If so, could you share the syntax you're trying to use?
$ docker exec -it citus_master psql -U postgres
psql (10.3 (Debian 10.3-1.pgdg90+1))
Type "help" for help.
postgres=# \dx
List of installed extensions
Name | Version | Schema | Description
---------+---------+------------+------------------------------
citus | 7.4-3 | pg_catalog | Citus distributed database
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(2 rows)
postgres=# show lc_collate ;
lc_collate
------------
en_GB.utf8
(1 row)
postgres=# show lc_ctype ;
lc_ctype
------------
en_GB.utf8
(1 row)
postgres=# show lc_messages ;
lc_messages
-------------
en_GB.utf8
(1 row)```
Hi @colton-citus ,
Many thanks for your help ! I will try with the Docker version that you have specified as well as with some other tool than pgAdmin. Also, will catch you up on the Slack channel with the modified config.
Thanks !