worker pod crashloopbackoff status

[TomBuyvoets]

** PLEASE ONLY USE THIS ISSUE TRACKER TO SUBMIT ISSUES WITH THE EXAMPLE VOTING APP **

• If you have a bug working with Docker itself, not related to these labs, please file the bug on the Docker repo *
• If you would like general support figuring out how to do something with Docker, please use the Docker Slack channel. If you're not on that channel, sign up for the Docker Community and you'll get an invite. *
• Or go to the Docker Forums *

Please provide the following information so we can assess the issue you're having

Description

All pods are running normal except the worker pod. After digging into the logs there is an exception of type AggregateException "no such device or address.

Steps to reproduce the issue, if relevant: 1. 2. 3.

Describe the results you received: worker pod in crashloopbackoff

Describe the results you expected: worker pod will run

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Output of docker info:

Containers: 17
 Running: 16
 Paused: 0
 Stopped: 1
Images: 8
Server Version: 18.09.6
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-50-generic
Operating System: Ubuntu 18.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.947GiB
Name: node1
ID: BZCL:OEV3:XFPE:CRZ3:ACL4:JE52:7RB4:IZYT:2G5S:BQVT:7QH7:AU27
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

Additional environment details (AWS, Docker for Mac, Docker for Windows, VirtualBox, physical, etc.):

environment: virtualbox vm ubuntu server v18.04

network card is configured as "network bridge adapter" all machines have an ip address in my own LAN.

output 'kubectl logs

System.AggregateException: One or more errors occurred. (No such device or address) ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No such device or address
   at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult)
   at System.Net.Dns.EndGetHostAddresses(IAsyncResult asyncResult)
   at System.Net.Dns.<>c.<GetHostAddressesAsync>b__14_1(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Npgsql.NpgsqlConnector.Connect(NpgsqlTimeout timeout)
   at Npgsql.NpgsqlConnector.RawOpen(NpgsqlTimeout timeout)
   at Npgsql.NpgsqlConnector.Open(NpgsqlTimeout timeout)
   at Npgsql.ConnectorPool.Allocate(NpgsqlConnection conn, NpgsqlTimeout timeout)
   at Npgsql.NpgsqlConnection.OpenInternal()
   at Worker.Program.OpenDbConnection(String connectionString) in /code/src/Worker/Program.cs:line 74
   at Worker.Program.Main(String[] args) in /code/src/Worker/Program.cs:line 19
---> (Inner Exception #0) System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No such device or address
   at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult)
   at System.Net.Dns.EndGetHostAddresses(IAsyncResult asyncResult)
   at System.Net.Dns.<>c.<GetHostAddressesAsync>b__14_1(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)<---

[BretFisher]

The .net worker is flaky on resolving redis and/or db. Try the java worker, which has the code in this repo but I don't think docker made an image for. I have one built at bretfisher/examplevotingapp_worker:java so just replace your worker image with that and it should work better

[BretFisher]

I tested this on Docker Desktop, and at first, the worker will crash because the db and redis aren't in DNS yet, but after a few minutes, it found DNS for db and redis and worked. Maybe you needed to wait a bit longer?

[rob2k40]

@BretFisher Still facing this issue, not able to find bretfisher/examplevotingapp_worker:java in Hub. Used your image bretfisher/examplevotingapp_worker, but it's still crashing for last 10 minutes. Tried multiple things like deleting all pods and services and starting DB before worker, but still facing issue. Please assist.

[BretFisher]

The java tag still exists on hub: https://hub.docker.com/r/bretfisher/examplevotingapp_worker/tags

[rob2k40]

Thank you so much @BretFisher for your valuable time. I tried your image and had earlier tried Java image of bgrissin/examplevotingapp_worker_java. All my services are up and running properly, but when I go to services and ingress --> Voting app service --> It opens the Interface to cast votes. Upon casting the vote, it gives below error:

500 Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Sorry to bother you again but it would be great if you could help last time.

[rob2k40]

@BretFisher Able to resolve 500 Internal server error by following this sequence: voting pod + service --> redis pod + service --> postgres pod + service --> worker --> result pod + service But now, voting page is accepting the request, but result page is not getting refreshed and showing 50-50 only. Anything I could do differently?

[rob2k40]

@BretFisher I used your vote, worker and result images also and used same sequence as earlier. Both voting and result pages are opening and voting page is accepting the request, but result page is not getting refreshed and showing 50-50 only.

[murugan425]

@BretFisher - same issue is still open, result page doesn't reflects the voting correctly. 50-50 always. Any assistance could be helpful.

[BretFisher]

We're sorry for the late response. Linux images have been refreshed, cleaned up, and are now multi-platform. If you're still having issues with the latest images, feel free to reopen this ticket.