bosh-deployment icon indicating copy to clipboard operation
bosh-deployment copied to clipboard
verified publisher icon cloudfoundry

Add create-env script for docker cpi

Open rkoster opened this issue 1 year ago • 2 comments

Apparently, I have had these changes in my local directory for a long time. Sharing so other people can use the docker cpi for bosh lite development.

rkoster avatar Nov 29 '24 08:11 rkoster

I tried this out and found following things:

On machines where people are not admin the CPI installation is an issue will fail with:

+ go build -o /Users/d047356/.bosh/installations/25678ae6-4d6f-4c9f-7901-e87e7e52363b/packages/docker_cpi/bin/cpi-darwin
failed to initialize build cache at /var/vcap/data/golang-1-darwin/cache: mkdir /var/vcap: permission denied
':

The BOSH Director deployment is failing reproducibly on my machine with:

...
  Updating instance 'bosh/0'... Finished (00:03:00)
  Waiting for instance 'bosh/0' to be running... Failed (00:05:34)
Failed deploying (00:13:13)
Cleaning up rendered CPI jobs... mmo:Finished (00:00:00)

The issue could be debugged with:

docker ps -a //find the container id
docker container exec -it <container-id-from-previous-cmd> /bin/bash

At the end all processes are running fine. It seems to be an issue with the time needed for all process to start. I found in the NATs logs following:

I, [2024-11-29T11:49:54.142163 #13286]  INFO : Nats Sync starting...
I, [2024-11-29T11:50:04.379177 #13286]  INFO : Executing NATS Users Synchronization
F, [2024-11-29T11:50:04.384810 #13286] FATAL : Failed to open TCP connection to 127.0.0.1:25555 (Connection refused - connect(2) for "127.0.0.1" port 25555)
F, [2024-11-29T11:50:04.384871 #13286] FATAL : /var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/net/http.rb:1603:in `initialize'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/net/http.rb:1603:in `open'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/net/http.rb:1603:in `block in connect'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/timeout.rb:196:in `timeout'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/net/http.rb:1601:in `connect'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/net/http.rb:1580:in `do_start'
/var/vcap/data/packages/director-ruby-3.2/cf38fddd06ffee4457ce716c183eed809abcb8e0/lib/ruby/3.2.0/net/http.rb:1569:in `start'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rest-client-2.1.0/lib/restclient/request.rb:727:in `transmit'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rest-client-2.1.0/lib/restclient/request.rb:163:in `execute'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rest-client-2.1.0/lib/restclient/request.rb:63:in `execute'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:107:in `call_bosh_api_no_auth'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:120:in `info'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:126:in `create_authentication_header'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:76:in `call_bosh_api'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:91:in `query_all_deployments'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:100:in `query_all_running_vms'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/users_sync.rb:20:in `execute_users_sync'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/runner.rb:46:in `sync_nats_users'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/bosh-nats-sync-0.0.0/lib/nats_sync/runner.rb:33:in `block in run'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:241:in `block in do_call'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler.rb:130:in `around_trigger'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:240:in `do_call'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:288:in `trigger_now'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:339:in `block (3 levels) in start_work_thread'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:342:in `block (2 levels) in start_work_thread'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:325:in `loop'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/rufus-scheduler-3.9.2/lib/rufus/scheduler/jobs_core.rb:325:in `block in start_work_thread'
/var/vcap/data/packages/nats/3ec477ff896abbf12eb13e87fa995ba9f6105d5e/gem_home/ruby/3.2.0/gems/logging-2.4.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
I, [2024-11-29T11:52:47.258933 #14396]  INFO : Nats Sync starting...
I, [2024-11-29T11:52:57.495151 #14396]  INFO : Executing NATS Users Synchronization

The step Adding Network Routes (sudo is required) also requires admin permissions.

beyhan avatar Nov 29 '24 12:11 beyhan

Additional context: https://github.com/cloudfoundry/bosh-deployment/pull/481

jpalermo avatar Dec 12 '24 15:12 jpalermo