Unable to push to registry: TLS handshake timeout

[kinghuang]

Description

Just getting back to Docker App after a long break. With 0.6.0, I'm unable to use docker-app push to push to my registry. Docker App just says TLS handshake timeout.

The registry's certificate is signed by GeoTrust. I can login to it using docker login …, and I can push and pull images with the docker CLI. But, docker-app doesn't work.

Steps to reproduce the issue:

1. Attempt to push a Docker App image to a private registry.

Describe the results you received:

▸ docker-app git:(master) docker-app push --namespace registry-dev.transzap.com/devops/templates/docker-app --repo app --tag test
Error: Get https://registry-dev.transzap.com/v2/: net/http: TLS handshake timeout

Describe the results you expected:

The app image should be pushed to the registry.

Additional information you deem important (e.g. issue happens only occasionally):

Using macOS 10.13.6. Running docker-app from within a container extending the docker image works.

Output of docker version:

Client: Docker Engine - Community
 Version:           18.09.0-ce-beta1
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        78a6bdb
 Built:             Thu Sep  6 22:41:53 2018
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0-ce-beta1
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       78a6bdb
  Built:            Thu Sep  6 22:49:35 2018
  OS/Arch:          linux/amd64
  Experimental:     true

Output of docker-app version:

Version:      v0.6.0
Git commit:   9f9c6680
Built:        Thu Oct  4 13:30:33 2018
OS/Arch:      darwin/amd64
Experimental: off
Renderers:    none

Output of docker info:

Containers: 41
 Running: 8
 Paused: 0
 Stopped: 33
Images: 211
Server Version: 18.09.0-ce-beta1
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
 NodeID: w97q3z0azzcv970g3e5hn9ski
 Is Manager: true
 ClusterID: o1cdcrn2etmq3kcz2aapse3n4
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 192.168.65.3
 Manager Addresses:
  192.168.65.3:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.125-linuxkit
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 7.786GiB
Name: linuxkit-025000000001
ID: 4IE3:LWO5:GQZ6:NP57:WDEP:C2HV:Y2LK:ZO7G:62CK:BKTF:3URM:W22G
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 90
 Goroutines: 248
 System Time: 2018-11-08T22:16:30.323729122Z
 EventsListeners: 6
HTTP Proxy: gateway.docker.internal:3128
HTTPS Proxy: gateway.docker.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, etc.):

[silvin-lubecki]

Hi @kinghuang , thank you for filling this issue! I think it might be related with #413 . Can you check if this comment helps you? I will then close this issue and track the fix on #413.

[kinghuang]

@silvin-lubecki I read through #413 before posting this issue, but I don't think it's the same thing. My registry (registry-dev.transzap.com) has a valid certificate chain, and doesn't use a self-signed certificate. docker-app works with it when I run it inside a container from the docker image, but not as a binary on my Mac. The docker CLI has no issues logging in and working with the registry.

Just to be sure, I've tried going to https://knowledge.digicert.com/solution/SO5761.html, downloading the GeoTrust Global CA and GeoTrust Primary Certification Authority – G3 certificates, and placing them in /usr/local/share/ca-certificates on my Mac. docker-app still reports TLS handshake timeout.

▸ docker-app git:(master) ls -al /usr/local/share/ca-certificates
total 16
drwxr-xr-x   4 king.huang  admin                   128  9 Nov 09:58 .
drwxrwxr-x  30 king.huang  admin                   960  9 Nov 09:50 ..
-rw-r--r--@  1 king.huang  TRANSZAP\Domain Users  1234  9 Nov 09:51 GeoTrust_Global_CA.pem
-rw-r--r--@  1 king.huang  TRANSZAP\Domain Users  1466  9 Nov 09:57 Geotrust_PCA_G3_Root.pem
▸ docker-app git:(master) docker-app push --namespace registry-dev.transzap.com/devops/templates/docker-app --repo app --tag test
Error: Get https://registry-dev.transzap.com/v2/: net/http: TLS handshake timeout

Is there a debug mode that can show more details about how docker-app is establishing the TLS connection?

[simonferquel]

The push/pull story is being reworked as part as moving to the CNAB runtime. I have no idea if/when it will fix the issue, but that is the reason we did not report back sooner on this. Sorry!

[vce-xx]

The push/pull story is being reworked as part as moving to the CNAB runtime. I have no idea if/when it will fix the issue, but that is the reason we did not report back sooner on this. Sorry!

Any update ?

[silvin-lubecki]

@kinghuang do you still have this issue with the latest release https://github.com/docker/app/releases/tag/v0.8.0 ?

[kinghuang]

I have not run into this issue with Docker App 0.8.0.

[vce-xx]

I am having a very similar issue. My registry has a valid certificate that is working nicely with docker image push.

However, with docker app push, I am getting "x509: certificate signed by unknown authority" :

$ docker app push hello --tag my.registry.com/hello:0.1.0
my.registry.com/hello:0.1.0-invoc
fixing up "my.registry.com/hello:0.1.0" for push: failed to resolve "my.registry.com/hello:0.1.0-invoc", push the image to the registry before pushing the bundle: failed to do request: Head https://my.registry.com/v2/hello/manifests/0.1.0-invoc: x509: certificate signed by unknown authority

$ docker app version 
Version:               v0.8.0
Git commit:            7eea32b7
Built:                 Tue Jun 11 20:53:26 2019
OS/Arch:               darwin/amd64
Experimental:          off
Renderers:             none
Invocation Base Image: docker/cnab-app-base:v0.8.0

@simonferquel Shall I open a different issue ?