kamal
kamal copied to clipboard
basecamp
App failed to boot. Options to increase healthcheck log lines?
I'm trying to setup kamal for the first time on my server, but it failed to boot. Unfortunately the log isn't helpful, and it's too short. Is there a way to show a more verbose logs? Currently deploying it via kamal deploy --verbose. Prior to the latest kamal, I can do this by setting healthcheck: log_lines: 10_000 based on https://kamal-deploy.org/v1/docs/configuration/healthchecks/#log-lines
INFO [ffaf045d] Running docker exec kamal-proxy kamal-proxy deploy application-web --target="a0fcb0ab2121:80" --host="application.com" --tls --deploy-timeout="30s" --drain-timeout="30s" --health-check-interval="5s" --health-check-timeout="30s" --health-check-path="/up" --target-timeout="60s" --buffer-requests --buffer-responses --log-request-header="Cache-Control" --log-request-header="Last-Modified" --log-request-header="User-Agent" on 5.223.54.19
DEBUG [ffaf045d] Command: docker exec kamal-proxy kamal-proxy deploy application-web --target="a0fcb0ab2121:80" --host="application.com" --tls --deploy-timeout="30s" --drain-timeout="30s" --health-check-interval="5s" --health-check-timeout="30s" --health-check-path="/up" --target-timeout="60s" --buffer-requests --buffer-responses --log-request-header="Cache-Control" --log-request-header="Last-Modified" --log-request-header="User-Agent"
DEBUG [ffaf045d] Error: target failed to become healthy within configured timeout (30s)
ERROR Failed to boot web on 5.223.54.19
INFO First web container is unhealthy on 5.223.54.19, not booting any other roles
INFO [72051b5b] Running docker container ls --all --filter name=^application-web-45e1731e49190f723282a72eb7cf0256450e7a1b$ --quiet | xargs docker logs --timestamps 2>&1 on 5.223.54.19
DEBUG [72051b5b] Command: docker container ls --all --filter name=^application-web-45e1731e49190f723282a72eb7cf0256450e7a1b$ --quiet | xargs docker logs --timestamps 2>&1
DEBUG [72051b5b] 2025-04-04T03:15:20.946327435Z I, [2025-04-04T03:15:20.829481 #7] INFO -- ddtrace: [ddtrace] DATADOG CONFIGURATION - CORE - {"date":"2025-04-04T03:15:20Z","os_name":"x86_64-pc-linux","version":"1.23.3","lang":"ruby","lang_version":"3.3.5","env":"production","service":"rails","dd_version":null,"debug":false,"tags":"env:production","runtime_metrics_enabled":false,"vm":"ruby-3.3.5","health_metrics_enabled":false,"profiling_enabled":false}
INFO [72051b5b] Finished in 0.130 seconds with exit status 0 (successful).
ERROR 2025-04-04T03:15:20.946327435Z I, [2025-04-04T03:15:20.829481 #7] INFO -- ddtrace: [ddtrace] DATADOG CONFIGURATION - CORE - {"date":"2025-04-04T03:15:20Z","os_name":"x86_64-pc-linux","version":"1.23.3","lang":"ruby","lang_version":"3.3.5","env":"production","service":"rails","dd_version":null,"debug":false,"tags":"env:production","runtime_metrics_enabled":false,"vm":"ruby-3.3.5","health_metrics_enabled":false,"profiling_enabled":false}
INFO [a6311a60] Running docker container ls --all --filter name=^application-web-45e1731e49190f723282a72eb7cf0256450e7a1b$ --quiet | xargs docker inspect --format '{{json .State.Health}}' on 5.223.54.19
DEBUG [a6311a60] Command: docker container ls --all --filter name=^application-web-45e1731e49190f723282a72eb7cf0256450e7a1b$ --quiet | xargs docker inspect --format '{{json .State.Health}}'
DEBUG [a6311a60] null
INFO [a6311a60] Finished in 0.139 seconds with exit status 0 (successful).
ERROR null
INFO [50001ee4] Running docker container ls --all --filter name=^application-web-45e1731e49190f723282a72eb7cf0256450e7a1b$ --quiet | xargs docker stop on 5.223.54.19
DEBUG [50001ee4] Command: docker container ls --all --filter name=^application-web-45e1731e49190f723282a72eb7cf0256450e7a1b$ --quiet | xargs docker stop
DEBUG [50001ee4] a0fcb0ab2121
INFO [50001ee4] Finished in 10.288 seconds with exit status 0 (successful).
Releasing the deploy lock...
DEBUG [679da6f3] Running /usr/bin/env rm .kamal/lock-application/details && rm -r .kamal/lock-application on 5.223.54.19
DEBUG [679da6f3] Command: /usr/bin/env rm .kamal/lock-application/details && rm -r .kamal/lock-application
DEBUG [679da6f3] Finished in 0.086 seconds with exit status 0 (successful).
Finished all in 306.8 seconds
ERROR (SSHKit::Command::Failed): Exception while executing on host 5.223.54.19: docker exit status: 1
docker stdout: Nothing written
docker stderr: Error: target failed to become healthy within configured timeout (30s)
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/command.rb:97:in `exit_status='
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/backends/netssh.rb:185:in `execute_command'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
<internal:kernel>:90:in `tap'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/backends/abstract.rb:80:in `execute'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/cli/app/boot.rb:3:in `execute'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/cli/app/boot.rb:57:in `start_new_version'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/cli/app/boot.rb:20:in `run'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/cli/app.rb:21:in `block (4 levels) in boot'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/cli/app.rb:20:in `each'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/cli/app.rb:20:in `block (3 levels) in boot'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/sshkit-1.23.2/lib/sshkit/backends/abstract.rb:31:in `run'
/Users/username/.local/share/mise/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/kamal-2.4.0/lib/kamal/sshkit_with_ext.rb:118:in `block (2 levels) in execute'
kamal deploy --verbose 6.50s user 15.95s system 7% cpu 5:07.22 total
The problem that causes this is not something that ends up in the logs of kamal, but in the logs of the application container. It might not be able to connect to your database for example.
I'm aware. Just that I'm trying to isolate the issue and having more log lines would be very helpful. Was able to do it in previous kamal 1.x version. Trying to find the same option in kamal 2.x.
Sorry, I didn't read your original message correctly. That seems like a useful feature to have in Kamal 2!