writing partitions : flushing parquet-buffer: open /tmp/phlaredb-parquet-buffers: read-only file system" in compactor

[Raghav65]

Describe the bug

We are using grafana/pyroscope version v1.2.0. We are running Pyroscope in microservice mode. All pods (distributor, ingester, store gateway, querier, compactor, query scheduler and query frontend) are running. Please find the pyroscope config in the attachment for all components. code : pyroscope_config.txt

Issue:

We are getting below error in compactor

ts=******* caller=bucket_compactor.go:xxx level=error component = compactor component = compactor tenant=***** msg="compaction job failed" duration=xxxxx duration_ms=xxxxxx err=" compact blocks [data-compactor/compact/--------------] writing partitions : flushing parquet-buffer: open /tmp/phlaredb-parquet-buffers: read-only file system"

compactor error looks similar to issue we raised previously 2651 hence I tried this solution in compactor - "you could also mount an emptyDir at /tmp in the container" @simonswine

Is that the right or permanent solution?

Environment

Infrastructure: Kubernetes (AKS) Deployment tool: Helm

[cyriltovena]

It's the right way to mitigate this for now. But we'll remove the need for that file soon and that shouldn't be require anymore.

We want to remove it because if Pyroscope crashloop, this will not be cleaned up.