Release file cleanup and standardisation

[sree-warrier]

Problem Statement

Where the release files are stored (Assuming it to the filestore(disk/buckets), please do correct if its are wrong) ? How the release files are been cleaned ? While checking we found that with the retention we set is not doing the cleanup activity with releases. Any configuration referring to the release retentions ? Any standard practice to follow on release file maintenance, as its filling the disk.

Solution Brainstorm

NA

[bijancot]

Hi,

Do you have any information which services/container that data filling up your disk? Long time ago i have issue with the postgresql even after execute cleanup manually. It turns out in my case i need to also do a vacuum in purpose to reclaim the disk.

and if you want to set the retention period you can take a look at .env https://github.com/getsentry/self-hosted/blob/master/.env#L3 but you need to rerun install process and docker compose up i think for better result

Cheers Baskoro

[webard]

Look at this issue https://github.com/getsentry/self-hosted/issues/2313

Make docker-compose.override.yml with content below:

x-restart-policy: &restart_policy
  restart: unless-stopped
services:
  disk-cleanup:
    <<: *restart_policy
    image: sentry-cleanup-self-hosted-local
    build:
      context: ./cron
      args:
        BASE_IMAGE: sentry-self-hosted-local
    entrypoint: "/entrypoint.sh"
    command: '"0 0 * * * find /_data/files -type d -empty | xargs -r rmdir"'
    volumes:
      - sentry-data:/_data

  disk-cleanup-files:
    <<: *restart_policy
    image: sentry-cleanup-self-hosted-local
    build:
      context: ./cron
      args:
        BASE_IMAGE: sentry-self-hosted-local
    entrypoint: "/entrypoint.sh"
    environment:
      # Leaving the value empty to just pass whatever is set
      # on the host system (or in the .env file)
      SENTRY_EVENT_RETENTION_DAYS:
    command: '"0 30 * * * find /_data/files -type f -atime +$SENTRY_EVENT_RETENTION_DAYS -delete"'
    volumes:
      - sentry-data:/_data

and run ./install.sh to install new services. They will clean outdated files and empty directories every day. You can also manually run commands from services to see effects immediately.

[sree-warrier]

@webard we are running the cron for 30 days also @bijancot we do run vaccum at the postgres level, but still we are able to see the OLDER app SDK releases which are nearly 60 days old. What could be the reasons ?

Also needed one clarification of this ask Where the release files are stored ?

[bijancot]

Hi @sree-warrier

I'am not 100% yet understand about what file stored where in sentry because some config will be stored to postgress and the flat file or data file of each container will saved under docker volume

but maybe the teams know how to find that @aldy505 @hubertdeng123

[aldy505]

Also needed one clarification of this ask Where the release files are stored ?

@sree-warrier it's stored on filesystem inside the sentry-data volume. You might want to read this part of the docs: https://develop.sentry.dev/self-hosted/experimental/external-storage/

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀