Otel-collector keeps crashing after docker install

[Thorfy]

Bug description

When installing signoz with docker compose, the otel-collector seems to restart again and again
Because of that no data is transfered to signoz.

its seems to be relied to SQL unknown DB due to user non write privilege. But i don't really understand the problem here he the part of the error stack i get when i inspect my otel-collector-1

2022-12-16 17:51:04 2022-12-16T16:51:04.779Z    info    service/telemetry.go:111        Setting up own telemetry...
2022-12-16 17:51:04 2022-12-16T16:51:04.779Z    info    service/telemetry.go:141        Serving Prometheus metrics      {"address": "0.0.0.0:8888", "level": "Basic"}
2022-12-16 17:51:04 2022-12-16T16:51:04.779Z    info    components/components.go:30     Stability level of component is undefined       {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "stability": "Undefined"}
2022-12-16 17:51:04 2022-12-16T16:51:04.893Z    info    clickhousetracesexporter/clickhouse_factory.go:122      View does not exist, skipping patch     {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "table": "dependency_graph_minutes_db_calls_mv"}
2022-12-16 17:51:04 2022-12-16T16:51:04.893Z    info    clickhousetracesexporter/clickhouse_factory.go:96       Running migrations from path:   {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "test": "/migrations"}
2022-12-16 17:51:04 2022-12-16T16:51:04.919Z    info    clickhousetracesexporter/clickhouse_factory.go:108      Clickhouse Migrate finished     {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "error": "code: 1001, message: std::__1::__fs::filesystem::filesystem_error: filesystem error: in rename: Permission denied [\"/var/lib/clickhouse/store/dc0/dc005ba1-50df-4910-bfd5-df9804273af5/tmp_insert_all_8_8_0\"] [\"/var/lib/clickhouse/store/dc0/dc005ba1-50df-4910-bfd5-df9804273af5/all_8_8_0/\"] in line 0: INSERT INTO schema_migrations (version, dirty, sequence) VALUES (1, 1, 1671209464898690188)"}
2022-12-16 17:51:05 Error: cannot build pipelines: failed to create "clickhousetraces" exporter, in pipeline "traces": code: 60, message: Table signoz_traces.distributed_signoz_index_v2 doesn't exist
2022-12-16 17:51:05 2022/12/16 16:51:05 application run finished with error: cannot build pipelines: failed to create "clickhousetraces" exporter, in pipeline "traces": code: 60, message: Table signoz_traces.distributed_signoz_index_v2 doesn't exist

here see the complete error stack in my container

Expected behavior

all cluster start

How to reproduce

1. install signoz with docker compose, docker desktop running
2. wait for container build
3. see the otel-collector keep crashing, but otel-collector-metrics is totally fine

Version information

• Signoz version: 0.12.0
• Browser version:
• Your OS and version: Windows with docker desktop, full running on wsl 2
• Your CPU Architecture(ARM/Intel): intel i5 13600K

Additional context

Fresh and updated installation , no incompatibility programs could interfer

[welcome[bot]]

Thanks for opening this issue. A team member should give feedback soon. In the meantime, feel free to check out the contributing guidelines.

[pranay01]

@Thorfy which version did you upgrade from?

[Thorfy]

@Thorfy which version did you upgrade from?

i don't have update from a previous version, its a fresh install from signoz 0.12.0 with docker compose install (not using swarm)

signoz: 0.12.0 signoz/signoz-otel-collector:0.66.0 Docker Desktop 4.15.0 Docker Engine v20.10.21 Docker Compose version v2.13.0

[prashant-shahi]

@Thorfy We don't officially support Windows.

I see that you have raised the PR to use volume instead of bind mount. That should resolve the issue in WSL.

[michalsteyn]

"We don't officially support Windows." That is a real bummer!!! I would massively upvote support for Docker in Windows.

[Krishnadas-KP]

@prashant-shahi , @Thorfy - I am running into this issue now with Debian on WSL2. Is there a configuration I can do to resolve this ?

[prashant-shahi]

@Krishnadas-KP You can switch from bind type volume mount to Docker Volume as done in the PR linked above.

https://github.com/SigNoz/signoz/pull/1869/files#diff-d37c6163aa120936d63ca42ef7de3a4f27f1d7c0e99878503b3e9aff1b68aecc

[gregorybleiker]

@prashant-shahi In your PR, you didn't reference the volume in the clickhouse service. - ./data/clickhouse/:/var/lib/clickhouse/ should be - data_clickhouse:/var/lib/clickhouse/

[abunet]

HI, any update on this?

Thanks

[aareman]

I have this issue on Ubuntu 22 (PopOS) with docker. (This could be a different issue, I have to check the logs of the containers next time this occurs)

For local development we have a local signoz running via docker compose. and 3 of the services go down frequently, not sure when, could be due to a reboot.

the services that get stuck in infinite restart are

1. alert manager
2. logspout
3. otel collector

Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Linux pop-os 6.4.6-76060406-generic #202307241739~1694621917~22.04~ac5e1a8 SMP PREEMPT_DYNAMIC Wed S x86_64 x86_64 x86_64 GNU/Linux