chroma-core
File handler's log file location?
I manage to run a chromadb container in a local k8s cluster. I have mounted an emptyDir volume and configured the file handler:
file:
formatter: default
class : logging.handlers.RotatingFileHandler
filename: chroma.log
Where is the full path of this chroma.log file and how to configure the location of it?
@khteh, which Chroma version are you using?
In pre-1.0 (0.6.x) versions using the logging configuration as you showed above will generally work.
For newer versions 1.0.x it logging doesn't technically work in the same way and you'd need an OTEL collector where to send the metrics + traces.
Let me know which version of Chroma you need help with.
STS spec:
image: chromadb/chroma:latest
Inside the pod:
root@chroma-0:/# chroma --version
chroma 1.0.0
hey @khteh, I've been doing some thinking and here's a conclusion I arrive at.
In cloud-native setting (k8s) it does not make sense to log to a file, especially not in the container. So what I'd suggest is for you to use either COTS (datadog, honeycomb, Grafana Cloud etc.) or an OSS solution (OTEL-Loki-Grafana). I'm happy to help you set things up in k8s.
Configure Logging + Telemetry in K8s for Chroma 1.0.5+
As of Chroma 1.0.5 we ship it with Rust-based server (CLI) so configuration is a bit different.
You will need to create a configmap with the following content:
########################
# HTTP server settings #
########################
open_telemetry:
service_name: "chroma"
endpoint: "http://<otel_collector_endpoint>:4317"
port: 8000
listen_address: "0.0.0.0"
####################
# General settings #
####################
persist_path: "/data"
And then you'll need to mount the above config to /config.yaml inside the chroma container. This will log some output to stdout and to OTEL collector (metrics and traces).
You sound very confident but I think you miss one thing - the EFK stack. So, this is not what I expect.
@khteh, far for me to sound confident, I'm merely suggesting an approach. EFK stack is perfectly valid way of logging too. For what is worth you can as easily swap the F in EFK stack with O thus EOK stack :D. Both OTEL collector and Fluentd essentially achieve the same goal, and it is more of a preference/experience with the stack. The only anti-pattern I see is the attempt to log to a file in container, especially when there's no rotation configured.
As mentioned above Chroma supports OTEL telemetry and for what I gather standard fluentd does not come with built-in OTEL support - https://docs.fluentd.org/input#list-of-input-plugins. If you are going into the EFK stack you may consider substituting Fluentd with Fluentbit as it comes with OTEL out of the box - https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry. Also Fluentbit can be configured via yaml which is generally simpler than fluentd config.
