bitnami
how to change zookeeper's logback.xml settings
Name and Version
bitnami/zookeeper:3.8.1
What architecture are you using?
None
What steps will reproduce the bug?
helm install zookeeper bitnami/zookeeper
Are you using any custom parameters or values?
no, I don't
What is the expected behavior?
I'd like to see the log file(zookeeper.log) under /opt/bitnami/zookeeper/logs directory but no files are found. It seems container default output is standard output.
I saw a post about changing the log4j config (ZOO_LOG4J_PROP = 'INFO, ROLLINGFILE') but in zookeeper version 3.8.1, logback was used not log4j, so it was not suitable. How can I leave log files under /opt/bitnami/zookeeper/logs?
What do you see instead?
/opt/bitnami/zookeeper/conf/logback.xml
<?xml version="1.0"?>
<!--
Copyright 2022 The Apache Software Foundation
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Define some default values that can be overridden by system properties
-->
<configuration>
<!-- Uncomment this if you would like to expose Logback JMX beans -->
<!--jmxConfigurator /-->
<property name="zookeeper.console.threshold" value="ERROR"/>
<property name="zookeeper.log.dir" value="."/>
<property name="zookeeper.log.file" value="zookeeper.log"/>
<property name="zookeeper.log.threshold" value="INFO"/>
<property name="zookeeper.log.maxfilesize" value="256MB"/>
<property name="zookeeper.log.maxbackupindex" value="20"/>
<!--
console
Add "console" to root logger if you want to use this
-->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>${zookeeper.console.threshold}</level>
</filter>
</appender>
<!--
Add ROLLINGFILE to root logger to get log file output
-->
<!--appender name="ROLLINGFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>${zookeeper.log.dir}/${zookeeper.log.file}</File>
<encoder>
<pattern>%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>${zookeeper.log.threshold}</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<maxIndex>${zookeeper.log.maxbackupindex}</maxIndex>
<FileNamePattern>${zookeeper.log.dir}/${zookeeper.log.file}.%i</FileNamePattern>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>${zookeeper.log.maxfilesize}</MaxFileSize>
</triggeringPolicy>
</appender-->
<!--
Add TRACEFILE to root logger to get log file output
Log TRACE level and above messages to a log file
-->
<!--property name="zookeeper.tracelog.dir" value="${zookeeper.log.dir}" />
<property name="zookeeper.tracelog.file" value="zookeeper_trace.log" />
<appender name="TRACEFILE" class="ch.qos.logback.core.FileAppender">
<File>${zookeeper.tracelog.dir}/${zookeeper.tracelog.file}</File>
<encoder>
<pattern>%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
</appender-->
<!--
zk audit logging
-->
<!--property name="zookeeper.auditlog.file" value="zookeeper_audit.log" />
<property name="zookeeper.auditlog.threshold" value="INFO" />
<property name="audit.logger" value="INFO, RFAAUDIT" />
<appender name="RFAAUDIT" class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>${zookeeper.log.dir}/${zookeeper.auditlog.file}</File>
<encoder>
<pattern>%d{ISO8601} %p %c{2}: %m%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>${zookeeper.auditlog.threshold}</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<maxIndex>10</maxIndex>
<FileNamePattern>${zookeeper.log.dir}/${zookeeper.auditlog.file}.%i</FileNamePattern>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>10MB</MaxFileSize>
</triggeringPolicy>
</appender>
<logger name="org.apache.zookeeper.audit.Slf4jAuditLogger" additivity="false" level="${audit.logger}">
<appender-ref ref="RFAAUDIT" />
</logger-->
<root level="INFO">
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
I have no name!@kafka-log-kafka-zookeeper-0:/opt/bitnami/zookeeper/logs$ ls -al total 0 drwxrwxr-x 2 root root 6 Jun 18 15:36 . drwxr-xr-x 1 root root 18 Jun 18 15:36 ..
Additional information
No response
Hi @wltinan,
Sorry for the delay, I wanted to perform some tests and ask the team about the issue. Unfortunately, I cannot see other way to do this for logback than adding logic to the chart/container in order to do that, or mounting a custom logback.xml file into the config directory directly.
Anyway, I consider that this should be supported by our container/chart logic a easy way to interact with the logging configuration file, so I will create a task for the team to implement this. We will reach you back here our workload allow us to address the task. Meanwhile, I will leave this issue marked as on-hold to prevent the stale bot to close it.
Hi. Hope all is well. I don't know how far ahead you are with solving this issue, but adding my comment here since I'm working on achieving JSON format structured logging from Zookeeper PODs.
I have filed an issue to the Zookeeper project (https://issues.apache.org/jira/browse/ZOOKEEPER-4757) to add the needed jar dependency and logback appender snippet, plus a way to change appender-ref by system property. But a way to provide a custom logback.xml in a CM is probably necessary for many log customization requirements, such as enabling audit logging.
~Unfortunately the response from zookeeper project is mixed so far, so I also filed a similar issue to the zookeeper Dockerfile maintainer in https://github.com/31z4/zookeeper-docker/issues/154 to get some feedback.~ UPDATE The Zookeeper project turned down the PR.
As I just realized that bitnami maintains their own Zookeeper Dockerfile, I filed an issue in that project for JSON logging feature: https://github.com/bitnami/containers/issues/51921.
This helm chart should of course allow users to completely replace logback.xml with their own custom one, but ideally they should then use some template in order to preserve controlling log level and json logging with env.vars.
We successfully deployed custom logback configuration with the bitnami ZK helm chart. Here's our approach:
We mounted a custom logback.xml via ConfigMap, which enables file-based logging:
# values.yaml
extraVolumes:
- name: custom-logback
configMap:
name: zookeeper-logback-config
extraVolumeMounts:
- name: custom-logback
mountPath: /opt/bitnami/zookeeper/conf/logback.xml
subPath: logback.xml
This approach provides full control over logging configuration. You can enable file appenders (commented out in the default logback.xml) by uncommenting and configuring the RollingFileAppender sections to write logs to /opt/bitnami/zookeeper/logs/.
This serves as a workaround until the chart natively supports toggling between stdout and file logging.
