dry icon indicating copy to clipboard operation
dry copied to clipboard

High CPU usage

Open 0x6D682D7331 opened this issue 7 months ago • 2 comments

Hi, I encountered high CPU usage with dry on small Docker Swarm cluster (2 nodes).

Without dry: Image

With dry:

Image

When i killed dry, journal showed a lot of entries like:

Mar 13 13:38:57  01-app-01-master dockerd[970]: 2025/03/13 13:38:57 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wra>
Mar 13 13:38:57  01-app-01-master dockerd[970]: time="2025-03-13T13:38:57.942650324+01:00" level=error msg="Handler for GET /v1.44/containers/json returned error: write unix /var/run/docker.sock->@: write: broken pip>
Mar 13 13:38:57  01-app-01-master dockerd[970]: 2025/03/13 13:38:57 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wra>
Mar 13 13:38:57  01-app-01-master dockerd[970]: time="2025-03-13T13:38:57.940368419+01:00" level=error msg="Handler for GET /v1.44/containers/json returned error: write unix /var/run/docker.sock->@: write: broken pip>

Dry version:

dry version 0.11.2, build fe820a70

Docker info:

Client: Docker Engine - Community
 Version:    27.5.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.20.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.4
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 128
  Running: 54
  Paused: 0
  Stopped: 74
 Images: 119
 Server Version: 27.5.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: active
  NodeID: gagjjfzq8t812yd0hkmd9zwaf
  Is Manager: true
  ClusterID: enkheij9i5vq273wmgyk6i13s
  Managers: 1
  Nodes: 2
  Default Address Pool: 10.0.0.0/8  
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 1
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 10.234.12.67
  Manager Addresses:
   10.234.12.67:2377
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.0-30-amd64
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.34GiB
 Name: app-01-master
 ID: 9ca1d0fc-e68c-4204-98ef-9065947c3dce
 Docker Root Dir: /mnt/why/not/secure
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Docker node:

 docker node ls
ID                            HOSTNAME                      STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
gagjjfzq8t812yd0hkmd9zwaf *   app-01-master   Ready     Active         Leader           27.5.1
q47f0ez3jbnvarvthh43jhtgy     app-02-master   Ready     Active                          27.5.1

0x6D682D7331 avatar Mar 13 '25 12:03 0x6D682D7331