Dawarich eats much diskspace /var/app/core (33 GB)

[tabacha]

OS & Hardware Linux Debian Dokcer

Version 0.25.4

Describe the bug Much Diskspace in Docker Overlay is used

root@docker:/srv/docker/dawarich# du -hs /var/lib/docker/overlay2/30e7f77078a813e5279b2143fa7315aecbd5567f8128eeb22be74e75b9fc4382/merged/var/app/core 
33G     /var/lib/docker/overlay2/30e7f77078a813e5279b2143fa7315aecbd5567f8128eeb22be74e75b9fc4382/merged/var/app/core
oot@docker:/srv/docker/dawarich# docker exec -it dawarich_sidekiq /bin/sh
/var/app # ls -la core 
-rw-------    1 root     root     35940139008 Apr 17 01:01 core

Expected behavior No Core File is writen.

Logs

docker compose logs dawarich_sidekiq |grep -B 65700 -A 5 'core dumped' |gzip > core.log.gz

See Attachemnt

core.log.gz

[Korenchkin]

the problem is not in writing core,but crashing (and writing core dump on crash),what you should do is provide some information about what platform you use(might be relevant),maybe some older docker or some older os?it might be even some library reading core dump,might be some points import problem(extract from your dump,if anyone can make something of it?): dawarich_sidekiq | D, [2025-04-17T01:00:16.932745 #8] DEBUG -- : [1m[35m (4.4ms)[0m [1m[34mSELECT ST_Distance(ST_GeomFromEWKT($1)::geography, ST_GeomFromEWKT($2)::geography)[0m [[nil, "POINT (9.861001046 53.47132802)"], [nil, "POINT (9.861097103 53.471328774)"]] dawarich_sidekiq | D, [2025-04-17T01:00:16.933683 #8] DEBUG -- : ↳ app/models/concerns/distanceable.rb:62:in 'block in Distanceable::ClassMethods#calculate_distance_for_array' dawarich_sidekiq | I, [2025-04-17T01:00:16.932995 #8] INFO -- : fail dawarich_sidekiq | W, [2025-04-17T01:00:16.934543 #8] WARN -- : {"context":"Job raised exception","job":{"retry":true,"queue":"reverse_geocoding","wrapped":"ReverseGeocodingJob","args":[{"job_class":"ReverseGeocodingJob","job_id":"7d7da4cc-4473-47a7-ad3e-9531e2e3001d","provider_job_id":null,"queue_name":"reverse_geocoding","priority":null,"arguments":["Point",7295425],"executions":0,"exception_executions":{},"locale":"en","timezone":"Europe/Berlin","enqueued_at":"2025-04-13T01:32:51.853147464Z","scheduled_at":null}],"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","jid":"b209ef539eae255825310a19","created_at":1744507971.8531938,"enqueued_at":1744842063.3566976,"error_message":"end of file reached","error_class":"EOFError","failed_at":1744507971.9207413,"retry_count":16,"retried_at":1744776367.2636678}} dawarich_sidekiq | W, [2025-04-17T01:00:16.934888 #8] WARN -- : EOFError: end of file reached dawarich_sidekiq | W, [2025-04-17T01:00:16.935629 #8] WARN -- : app/services/reverse_geocoding/points/fetch_data.rb:21:in 'ReverseGeocoding::Points::FetchData#update_point_with_geocoding_data' dawarich_sidekiq | app/services/reverse_geocoding/points/fetch_data.rb:15:in 'ReverseGeocoding::Points::FetchData#call' dawarich_sidekiq | app/jobs/reverse_geocoding_job.rb:11:in 'ReverseGeocodingJob#perform' dawarich_sidekiq | D, [2025-04-17T01:00:16.934732 #8] DEBUG -- : [1m[35m (3.8ms)[0m [1m[34mSELECT ST_Distance(ST_GeomFromEWKT($1)::geography, ST_GeomFromEWKT($2)::geography)[0m [[nil, "POINT (9.898870401 53.471512171)"], [nil, "POINT (9.898768896 53.471532203)"]] dawarich_sidekiq | D, [2025-04-17T01:00:16.936874 #8] DEBUG -- : ↳ app/models/concerns/distanceable.rb:62:in 'block in Distanceable::ClassMethods#calculate_distance_for_array' dawarich_sidekiq | /var/app/vendor/bundle/ruby/3.4.0/gems/sidekiq-7.3.9/lib/sidekiq/component.rb:17: [BUG] heap_page_body_free: munmap failed dawarich_sidekiq | ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [x86_64-linux-musl]

[Korenchkin]

try latest update 0.25.6,maybe it fixes your problem too(maybe in a few minutes,docker not up yet,look for version on your webpage when started) there should be some fix for incomplete import,might be your crash problem?start new/clean if you can

[tabacha]

I think there are two Problems. One is that the instance is crashing. The other is, that it is writing a core file. The problem with crashing is not a big one for me, because of health monitor in docker etc. is restarting sidekiq and it happens only once a week or even less than that. But the storage usage is a problem for me.

I asked chatgpt and it supposed to change the compose file:

services:
  ...:
    ulimits:
      core:
        soft: 0
        hard: 0

Perhabs we should change it?