JFrog CLI net/http: TLS handshake timeout
Hello,
We have a situation where we throttle requests to Artifactory through NGINX, if requests are coming too rapidly.
However, it seems that this throttling sometimes causes issues when using the jf c add command, with the error:
[Warn] (Attempt 1) - Failure occurred while sending GET request to https://artifactory.com/api/security/encryptedPassword: Get "https://artifactory.com/api/security/encryptedPassword": net/http: TLS handshake timeout
[Info] executor timeout after 0 attempts with 0 milliseconds wait intervals
[Info] Trace ID for JFrog Platform logs: aadd224d84634252
[Error] The following error was received while trying to encrypt your password: Get "https://artifactory.com/api/security/encryptedPassword": net/http: TLS handshake timeout
We would prefer not to disable the TLS verification.
Is it possible to increase the timeout threshold for JFrog CLI?
Best regards, David
So we did some digging around.
Neither NGINX nor Artifactory recognized any issues in their log files. No errors or even warnings. Not even any info messages about this.
We tried to increase the flow through NGINX which lowered the frequency of this issue.
So it seems like an issue entirely related to JFrog CLI. We also noticed the time stamps. It seems JFrog CLI waits about 10 seconds for the handshake before giving up.
However - even when disabling the TLS verification this error still occurred, which leads me to believe it is a general timeout issue for JFrog CLI.
My best guess is, that JFrog CLI does the following (very crudely set up):
- Starts a timer for 10 seconds
- Tries the TLS handshake
- The request is throttled by load balancer.
- Instead of waiting for the response, JFrog CLI immediately tries again (or very shortly after)
- The new request is throttled once again, and the steps are repeated over and over.
- The request is throttled by load balancer.
- As long as the load balancer is throttling the requests, JFrog CLI will eventually fail, when the 10 seconds are over, because it keeps sending new requests instead of waiting for a reply (or only waits for a very short duration).
This is just conjecture on my part, but based on the following observations:
- At no time was there a "delay" of 10 seconds when throttling occurred in NGINX. Longest wait time there was was 3 seconds.
- This is inconsistent when JFrog CLI would fail after 10 seconds of waiting time, since the delay was never that big.
- Increasing the rate limit of NGINX lowered the chance of JFrog CLI failing
- No server recognized an issue existing. The issue is entirely client side.
- Parallel instances of
jf c addcould pass while some would fail.
It is entirely jf c add that fails. Any other JFrog CLI command (as observed) have no issue with the throttling.
Finally, this has not been an issue before, but started appearing about 2 months ago.
I hope the above can help others with this issue or help you maybe pinpointing the source of the issue.
Hi @DavidRadoorHummel ,
Unfortunately there isn't a direct command-line flag to adjust this specific timeout, we request that you submit a feature request ticket. This will enable our engineering team to formally track and prioritize a solution for this limitation in a future release, and keep you updated on its progress.
Feedback provided