tiny-maven-proxy icon indicating copy to clipboard operation
tiny-maven-proxy copied to clipboard

Published 20 hours ago • verified publisher icon timboudreau

Repository redirects cause tiny-maven-proxy to hang

Open mbafford opened this issue 5 years ago • 4 comments

It looks like tiny-maven-proxy has a bug which causes the request to hang for 30+ minutes (indefinitely?) when the repository returns a 301/302 redirect.

Using the latest JAR build linked from this repository:

java -jar ./tiny-maven-proxy.jar --maven.proxy.debug true --log.console true --mirror https://oss.sonatype.org/content/repositories/releases/
START WITH URLS
{"msg":"config","debug":true,"level":30,"download.threads":"24","pid":94942,"dir":"/var/folders/vb/v36bb1js3dbfhkj9l7_2n3440000gn/T/maven","eventThreads":"3","hostname":"agentsmith.local","v":0,"name":"startup","time":"2020-02-20T14:05:15.345Z","mirroring":["https://oss.sonatype.org/content/repositories/releases/"],"workers":"6"}
TinyMavenProxy 1.5 on port  5956 serving:
                            Repo:          https://oss.sonatype.org/content/repositories/releases/
Settings:
download.threads            24
workers                     6
eventThreads                3

Called with curl:

$ curl -f -v -O http://10.10.11.10:5956/BAD.jar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.10.11.10...
* TCP_NODELAY set
* Connected to 10.10.11.10 (10.10.11.10) port 5956 (#0)
> GET /BAD.jar HTTP/1.1
> Host: 10.10.11.10:5956
> User-Agent: curl/7.54.0
> Accept: */*
>
  0     0    0     0    0     0      0      0 --:--:--  0:34:12 --:--:--     0^C

Logs for those 34 minutes:

  defer and download  BAD.jar
attempt https://oss.sonatype.org/content/repositories/releases/BAD.jar
state Connecting https://oss.sonatype.org/content/repositories/releases/BAD.jar
state Connected https://oss.sonatype.org/content/repositories/releases/BAD.jar
state SendRequest https://oss.sonatype.org/content/repositories/releases/BAD.jar
state AwaitingResponse https://oss.sonatype.org/content/repositories/releases/BAD.jar
state Redirect https://oss.sonatype.org/content/repositories/releases/BAD.jar
state Connecting https://oss.sonatype.org/content/repositories/releases/BAD.jar

And then after killing the curl process:

Complete w/ remaining 1
state Cancelled https://oss.sonatype.org/content/repositories/releases/BAD.jar

The URL being fetched ( https://oss.sonatype.org/content/repositories/releases/BAD.jar ) returns a 302 redirect very quickly when called directly with curl:

$ curl -v 'https://oss.sonatype.org/content/repositories/releases/BAD.jar'
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 302
< server: awselb/2.0
< date: Thu, 20 Feb 2020 14:44:34 GMT
< content-type: text/html
< content-length: 126
< location: https://repo1.maven.org:443/content/repositories/releases/BAD.jar
<
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
</body>
</html>

This seems to be related to the repository returning a redirect. I can reproduce with https://repo1.maven.org/content/repositories/releases/, which 301 redirects to https://repo1.maven.org/maven2/BAD.jar in the above test.

However, a repository which doesn't do redirects fails quickly:

$ java -jar ./tiny-maven-proxy.jar --maven.proxy.debug true --log.console true --mirror https://repo1.maven.org/maven2/
START WITH URLS
{"msg":"config","debug":true,"level":30,"download.threads":"24","pid":95653,"dir":"/var/folders/vb/v36bb1js3dbfhkj9l7_2n3440000gn/T/maven","eventThreads":"3","hostname":"agentsmith.local","v":0,"name":"startup","time":"2020-02-20T14:47:29.647Z","mirroring":["https://repo1.maven.org/maven2/"],"workers":"6"}
TinyMavenProxy 1.5 on port  5956 serving:
                            Repo:          https://repo1.maven.org/maven2/
Settings:
download.threads            24
workers                     6
eventThreads                3

  defer and download  BAD.jar
attempt https://repo1.maven.org/maven2/BAD.jar
state Connecting https://repo1.maven.org/maven2/BAD.jar
state Connected https://repo1.maven.org/maven2/BAD.jar
state SendRequest https://repo1.maven.org/maven2/BAD.jar
state AwaitingResponse https://repo1.maven.org/maven2/BAD.jar
   download failed:  https://repo1.maven.org/maven2/BAD.jar, 404 Not Found
state Cancelled https://repo1.maven.org/maven2/BAD.jar
{"msg":"oneDownloadFailed","hostname":"agentsmith.local","u":"https://repo1.maven.org/maven2/BAD.jar","level":30,"v":0,"name":"download","pid":95653,"time":"2020-02-20T14:47:46.868Z","dlid":"k6uv0skq:0"}
 fail  404 Not Found
{"msg":"allDownloadsFailed","path":"BAD.jar","hostname":"agentsmith.local","u":"https://repo1.maven.org/maven2/BAD.jar","level":30,"v":0,"name":"download","pid":95653,"id":"eafyv:0","time":"2020-02-20T14:47:46.869Z","status":{"reason":"Not Found","code":404}}
state HeadersReceived https://repo1.maven.org/maven2/BAD.jar
Status https://repo1.maven.org/maven2/BAD.jar 404 Not Found
   download failed:  https://repo1.maven.org/maven2/BAD.jar, 404 Not Found
{"dur":611,"msg":"request","agent":"curl/7.54.0","address":"10.10.11.10","method":"GET","level":20,"pid":95653,"path":"BAD.jar","hostname":"agentsmith.local","v":0,"host":"10.10.11.10:5956","name":"requests","id":"eafyv:0","time":"2020-02-20T14:47:46.880Z","status":404}
Complete w/ remaining 0

mbafford avatar Feb 20 '20 14:02 mbafford

@mbafford I'm also seeing this issue. Did you figure anything out? In general it would be nice to set a timeout. Maybe that can be done by putting a proxy like nginx in front of this and set the timeout there.

alexklibisz avatar May 01 '20 22:05 alexklibisz

@alexklibisz Off-topic for this issue, so I'll keep it brief here (more comments at the link below). I found it easier to scratch my itch by writing a quick Python script (https://gist.github.com/mbafford/d25939a35f5066d46753db6bfba7684b) to do the mirroring from my local ~/.m2/repository

mbafford avatar May 09 '20 23:05 mbafford

@alexklibisz Off-topic for this issue, so I'll keep it brief here (more comments at the link below). I found it easier to scratch my itch by writing a quick Python script (https://gist.github.com/mbafford/d25939a35f5066d46753db6bfba7684b) to do the mirroring from my local ~/.m2/repository

Neat, thanks. We ended up going in a different direction. We set up a virtual repository in JFrog to merge all of our internal and external repos, deployed an nginx reverse proxy with caching in our cluster so that it proxied requests out to Jfrog, and pointed our builds at the nginx server. I did experiment with a pure nginx reverse proxy solution but it turns out to be very difficult to handle all of the response codes and possible errors with the different artifact repos.

alexklibisz avatar May 11 '20 01:05 alexklibisz

I am also seeing this issue with a 303.

curl -v https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
*snip*
> GET /m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom HTTP/1.1
> Host: plugins.gradle.org
> User-Agent: curl/7.64.1
> Accept: */*
> 


tiny-maven-proxy output:


  defer and download  org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
attempt https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connecting https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connected https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state SendRequest https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state AwaitingResponse https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Redirect https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connecting https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
Complete w/ remaining 1
state Cancelled https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
  defer and download  org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
attempt https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connecting https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connected https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state SendRequest https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state AwaitingResponse https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Redirect https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connecting https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
Complete w/ remaining 1
state Cancelled https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
  defer and download  org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
attempt https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connecting https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connected https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state SendRequest https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state AwaitingResponse https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Redirect https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
state Connecting https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom
Complete w/ remaining 1
state Cancelled https://plugins.gradle.org/m2/org/jlleitschuh/gradle/ktlint-gradle/9.4.1/ktlint-gradle-9.4.1.pom

saramah avatar Jan 28 '22 23:01 saramah