Clamav with persistent storage doesn't clean up tmp files

[mayasBK]

Hi,

I use the clamav helm chart in version 3.7.1 and have it configured with persistent storage (2 gb sized pvc).

My problem is that freshclam creates tmp files during the update but doesn't clean them up. After a week the pvc runs out of space and clamav fails to update. This breaks the whole service.

mounted pvc in clamav:

/data $ du -hs *
276.0K    bytecode.cvd
196.4M    daily.cld
4.0K    freshclam.dat
16.0K    lost+found
162.6M    main.cvd
392.9M    tmp.01c1665ebb
392.9M    tmp.248f9586f1
392.9M    tmp.da754e1821
4.0K    tmp.e21c461387

My values.yaml:

---
clamav:
  kind: StatefulSet
  replicaCount: 1

  persistentVolume:
    enabled: true
    size: 2Gi
    accessModes:
      - ReadWriteOnce

  resources:
    limits:
      memory: 2Gi
    requests:
      cpu: 100m
      memory: 2Gi

  hpa:
    enabled: false

  startupProbe:
    initialDelaySeconds: 60
    periodSeconds: 30
    failureThreshold: 3
    timeoutSeconds: 1

  livenessProbe:
    initialDelaySeconds: 300
    periodSeconds: 10
    failureThreshold: 3
    timeoutSeconds: 1

  readinessProbe:
    initialDelaySeconds: 90
    periodSeconds: 10
    failureThreshold: 3
    timeoutSeconds: 1


  freshclamConfigDict:
    DatabaseDirectory: /data
    LogTime: "yes"
    PidFile: /tmp/freshclam.pid
    DatabaseOwner: "100"
    DatabaseMirror: database.clamav.net
    ScriptedUpdates: "yes"
    Bytecode: "yes"

  clamdConfigDict:
    DatabaseDirectory: /data
    TemporaryDirectory: /tmp
    LogTime: "yes"
    LogVerbose: "yes"
    PidFile: /tmp/clamd.pid
    LocalSocket: /tmp/clamd.sock
    TCPSocket: 3310
    Foreground: "yes"
    DetectPUA: "yes"
    HeuristicAlerts: "yes"
    Bytecode: "yes"
    ScanPE: "yes"
    ScanELF: "yes"
    ScanOLE2: "yes"
    ScanPDF: "yes"
    ScanSWF: "yes"
    ScanMail: "yes"
    PhishingSignatures: "yes"
    ScanHTML: "yes"
    ScanArchive: "yes"
    ScanXMLDOCS: "yes"
    AlgorithmicDetection: "yes"
    PhishingAlwaysBlockCloak: "yes"
    PhishingAlwaysBlockSSLMismatch: "yes"
    StructuredDataDetection: "yes"
    MaxScanSize: 150M
    MaxFileSize: 30M
    MaxRecursion: 16
    MaxFiles: 15000
    MaxScanTime: 120000

Maybe someone knows this issue and how I can workaround it.

[dramaturg]

I stumbled over this as well and worked around the issue by patching in a sidecar to clean up older tmp files:

      - image: clamav/clamav:1.4.3_base
        imagePullPolicy: IfNotPresent
        name: sidecar
        command:
        - /bin/sh
        - -c
        args:
        - |
          while true; do
            find /data -maxdepth 1 -type d -name tmp.\* -mtime +2 -print0 | xargs -0 rm -fr
            sleep 1h
          done
        resources:
          requests:
            cpu: 10m
            memory: 16Mi
        volumeMounts:
        - mountPath: /data
          name: clamav-data

I don't know if this is the right thing to do but it works for me. If there is no more elegant way around this (does clamd really usually leave all this c**p around?) something like this could be incorporated in the helm chart?

[desaintmartin]

Oh, this is weird. Since we use it in a stateless way with "frequent" (~every week) restart, we haven't seen this. I suppose we could add this to the chart, PR accepted, even if it's dirty as hell. But i'm reading that it may occur because it lacks resources...

[mayasBK]

@desaintmartin may I ask...did you adjust any timeouts for startup probes? Your link seems to confirm that tmp files are left behind if the update process is not finished. Due to out of memory or maybe also when it doesn't finish in time and is killed by timeouts.