github-workflows-kt icon indicating copy to clipboard operation
github-workflows-kt copied to clipboard

[Bug] Checksum for Maven-based bindings is incorrect

Open krzema12 opened this issue 1 year ago • 1 comments

Action

Consuming the Maven-based bindings, see e.g. this job run: https://github.com/typesafegithub/github-workflows-kt/actions/runs/9142688602/job/25138781613#step:6:157

Expected

There are no warnings related to incorrect checksum.

Actual

A warning is printed for each action. E.g.

[main] WARN org.jetbrains.kotlin.org.eclipse.aether.internal.impl.WarnChecksumPolicy - Could not validate integrity of download from https://github-workflows-kt-bindings.colman.com.br/binding/actions/setup-python/v5/setup-python-v5.jar
org.jetbrains.kotlin.org.eclipse.aether.transfer.ChecksumFailureException: Checksum validation failed, expected 'e2c2511fe50fceda4868994bc9913787' (REMOTE_EXTERNAL) but is actually 'c1f543329a15c8a62e5edc157b2ff27e'

Workaround, if exists

None, it's non-blocking - the compilation goes on.

Library version

Not related to the library, it comes from the server.

krzema12 avatar May 18 '24 21:05 krzema12

We calculate the checksums here: https://github.com/typesafegithub/github-workflows-kt/blob/fa2eb91b41615308b600fcd04fa914000ef55f9f/maven-binding-builder/src/main/kotlin/io/github/typesafegithub/workflows/mavenbinding/VersionArtifactsBuilding.kt#L18 Some time ago, it worked fine. I first noticed it in this job run: https://github.com/typesafegithub/github-workflows-kt/actions/runs/9101024130/job/25017188085

krzema12 avatar May 18 '24 21:05 krzema12

Likely caused by https://github.com/LeoColman/MyStack/commit/e22d71a3d984345fc84aca12bd9636777c839f38. Increasing the number of replicas introduced an inconsistency in how JARs are generated, likely in timestamps and/or file order.

krzema12 avatar May 26 '24 06:05 krzema12

Does it still happen? Shouldn't md5 return the same given the same input? How urgent is this?

LeoColman avatar May 27 '24 01:05 LeoColman

@LeoColman no worries, this is not actionable for you, it's not a problem with your hosting directly 😊 It's just after increasing the number of instances revealed technically a bug in the service's logic. The bug is not serious, it's just when using the bindings, there's a long stack trace printed when executing the script, but the script doesn't fail. It just looks scary and very noisy in the terminal. I have a plan how to fix it.

Yes, the same MD5 should be returned but it's different likely because of file timestamps timestamps in the JAR, and maybe non-deterministic order of entries in the JAR.

krzema12 avatar May 27 '24 13:05 krzema12

Let me confirm my hypothesis regarding the root cause. Requesting a JAR two minutes apart from each other - the timestamps are different:

% zipinfo checkout-v4-23-14.jar 
Archive:  checkout-v4-23-14.jar
Zip file size: 10004 bytes, number of entries: 5
-rw----     2.0 fat     1477 bl defN 24-Jun-08 23:14 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth.class
-rw----     2.0 fat     1062 bl defN 24-Jun-08 23:14 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth$Infinite.class
-rw----     2.0 fat      940 bl defN 24-Jun-08 23:14 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth$Value.class
-rw----     2.0 fat    25233 bl defN 24-Jun-08 23:14 io/github/typesafegithub/workflows/actions/actions/Checkout.class
-rw----     2.0 fat       24 bl defN 24-Jun-08 23:14 META-INF/main.kotlin_module
5 files, 28736 bytes uncompressed, 8852 bytes compressed:  69.2%
% zipinfo checkout-v4-23-16.jar 
Archive:  checkout-v4-23-16.jar
Zip file size: 10004 bytes, number of entries: 5
-rw----     2.0 fat     1477 bl defN 24-Jun-08 23:16 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth.class
-rw----     2.0 fat     1062 bl defN 24-Jun-08 23:16 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth$Infinite.class
-rw----     2.0 fat      940 bl defN 24-Jun-08 23:16 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth$Value.class
-rw----     2.0 fat    25233 bl defN 24-Jun-08 23:16 io/github/typesafegithub/workflows/actions/actions/Checkout.class
-rw----     2.0 fat       24 bl defN 24-Jun-08 23:16 META-INF/main.kotlin_module
5 files, 28736 bytes uncompressed, 8852 bytes compressed:  69.2%

krzema12 avatar Jun 08 '24 21:06 krzema12

After introducing determinstic entry order and resetting the timestamps:

% zipinfo checkout-v4-23-32.jar 
Archive:  checkout-v4-23-32.jar
Zip file size: 10134 bytes, number of entries: 5
-rw----     2.0 fat       24 bX defN 70-Jan-01 01:00 META-INF/main.kotlin_module
-rw----     2.0 fat     1477 bX defN 70-Jan-01 01:00 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth.class
-rw----     2.0 fat     1062 bX defN 70-Jan-01 01:00 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth$Infinite.class
-rw----     2.0 fat      940 bX defN 70-Jan-01 01:00 io/github/typesafegithub/workflows/actions/actions/Checkout$FetchDepth$Value.class
-rw----     2.0 fat    25233 bX defN 70-Jan-01 01:00 io/github/typesafegithub/workflows/actions/actions/Checkout.class
5 files, 28736 bytes uncompressed, 8852 bytes compressed:  69.2%

krzema12 avatar Jun 08 '24 21:06 krzema12

The bug should be gone, let's wait for several commits to confirm.

krzema12 avatar Jun 08 '24 21:06 krzema12