jib
jib copied to clipboard
GoogleContainerTools
Cannot upgrade Google HTTP libraries
#3415 downgraded Google HTTP libraries to resolve #3058 and #3409. However, we cannot delay upgrading libraries indefinitely.
Moreover, even if we downgraded them, Gradle and Maven can still pull in newer versions depending on how a project is set up. Examples:
- https://github.com/GoogleContainerTools/jib/issues/3409#issuecomment-902360557
- https://github.com/GoogleContainerTools/jib/issues/3058#issuecomment-900681904
- https://github.com/googleapis/google-http-java-client/issues/1409#issuecomment-879819285I
In the last case, we didn't release jib-core with the problematic Google HTTP library version, but Quarkus picked up the latest version; I think jib-core will be more susceptible to version overriding than Jib plugins.
These issues need to be investigated further before we can upgrade the libraries.
I assume the challenge is that we cannot reproduce the problem. Is this correct?
For #3409, I think I know what exactly is the problem, but ~~I have trouble reproducing this~~. Filed a fix (https://github.com/googleapis/google-http-java-client/pull/1427) against the library.
For #3058, yeah, I haven't been able to reproduce it. And the root cause is still unknown. It may or may not be a fault of Amazon.
In Quarkus, we just ran into a dependency "conflict" because of this: https://github.com/quarkusio/quarkus/pull/20507
I anyone looking at getting the remaining issues fixed upstream? I understand it's hard to reproduce?
#3409, which caused an outright no-go friction to many people with the "premature end of chunk" error, has been fixed upstream. ~~New versions of the Google HTTP libraries with the fix are live now.~~
The other issue, #3058, is a very subtle one. It seems to happen to only some small group of people specifically using AWS ECR. Also, I've seen elsewhere that people run into the failure out of the Jib context as well, so it seems like a general problem. It's unclear if it's the fault of AWS or the Apache HttpClient. It's hard to reproduce, and I asked if anyone seeing the failure can contribute their time to debug it with us, but so far, no one responded.
Therefore, for Quarkus, IMO it'd be acceptable to upgrade the Google HTTP libraries to the latest once new versions are released.
Thanks @chanseokoh, just one question:
New versions of the Google HTTP libraries with the fix are live now.
Are you sure about that? 1.40.0 does not seem to include https://github.com/googleapis/google-http-java-client/pull/1427?
Sorry, I was dumb. The fix was merged on Sep 23, while 1.40.0 was released on Aug 26. We still have to wait for a new release. Sorry for the misinformation.
@chanseokoh alright, no worries!
Apart from those known issues in newer Google HTTP libs, would anything break if using latest jib-core with 1.38.0 of Google HTTP?
@famod apart from those listed here (one of which is critical), I'm not aware of anything else. But generally speaking, there's always a chance of behavior changes whenever you upgrade. But in general, definitely one needs to keep libraries up-to-date.
There were 3 issues that blocked upgrade:
- premature end of chunk coded message body -- fixed in googleapis/google-http-java-client#1427
- certificate issue with docker containers stored on AWS S3 https://github.com/aws/aws-sdk-java/issues/1919 https://github.com/aws/aws-sdk-java-v2/issues/1786
In any case, Jib can't hold off upgrading core HTTP libraries indefinitely. We should upgrade, and then help users work with AWS support if necessary.
jib-core 0.23.0, jib-maven-plugin 3.3.1, jib-gradle-plugin 3.3.1, and jib-cli 0.12.0 have been released with the HTTP libraries upgraded.
