jq: fix hash mismatch problem

[bradfordzhang]

Maintainer: @ratkaj Compile tested: x86 openwrt 22.03.0 Run tested: x86 openwrt 22.03.0 test done

Description: change jq source from github to openwrt source, because openwrt hacked/changed jq

#lede10195 mentioned this problem.

[Ansuel]

@bradfordzhang this happen when someone force push stuff to the repo and we have same release but different tar... Something strange is happening there...

[bradfordzhang]

@bradfordzhang this happen when someone force push stuff to the repo and we have same release but different tar... Something strange is happening there...

i tested it. github package hash does not match with openwrt source.

[bradfordzhang]

jq-1.6.tar(1).gz from https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz jq-1.6.tar(2).gz from https://sources.cdn.openwrt.org/jq-1.6.tar.gz jq-1.6.tar(3).gz from https://sources.openwrt.org/jq-1.6.tar.gz

[Ansuel]

@bradfordzhang very suspicious not wired

[bradfordzhang]

@bradfordzhang this happen when someone force push stuff to the repo and we have same release but different tar... Something strange is happening there...

can we set two hash so that any one of the two hash match can pass the test.

[bradfordzhang]

@bradfordzhang very suspicious not wired

my poor english😂

[Ansuel]

can we set two hash so that any one of the two hash match can pass the test.

This is a problematic situation honestly... and actually a security risk of some sort... If you want the direct answer NO.

We can't have 2 hash for the same package... This is because one release have an unique hash and if that changes there could be only 2 reason:

• the download is corrupted
• someone did something strange and the package changed silently

Now we don't know what has changed and sometimes this is really bad... (push a clean zip... then modify it and insert virus in it and propose the same release like nothing happen)

In short the package maintainer should handle this and decide what should be done... force push should never be used and in any case a release should not be force pushed to prevent these kind of security concern.

[bradfordzhang]

can we set two hash so that any one of the two hash match can pass the test.

This is a problematic situation honestly... and actually a security risk of some sort... If you want the direct answer NO.

We can't have 2 hash for the same package... This is because one release have an unique hash and if that changes there could be only 2 reason:

• the download is corrupted
• someone did something strange and the package changed silently

Now we don't know what has changed and sometimes this is really bad... (push a clean zip... then modify it and insert virus in it and propose the same release like nothing happen)

In short the package maintainer should handle this and decide what should be done... force push should never be used and in any case a release should not be force pushed to prevent these kind of security concern.

you're right. but jq has force pushed tag. that causes problem😂

[bradfordzhang]

@Ansuel hi! should aria2c continue download from next mirror when hash mismatch?

[bradfordzhang]

@Ansuel can we update cdn jq package to github version?

[neheb]

@bradfordzhang this happen when someone force push stuff to the repo and we have same release but different tar... Something strange is happening there...

this tarball is not a github generated one. Still weird.

[1715173329]

By checking the date, you can see the author re-uploaded the archives 18 days after the release, but 0099bad30b22fe895223393bcc60f44b30fa86f9 was commited just 4 days after.

So this outdated hash in Makefile has been kept for near 4 years! And even nobody noticed it as download.pl would automatically fallback to OpenWrt's mirror host which stored the old archive until recently aria2c support broke this feature.

Compared to old archive, the new one added missing git submodule which won't be packed by GitHub's tarball API:

$ git diff --stat --no-index jq-1.6-old jq-1.6
/dev/null => jq-1.6/modules/oniguruma/.git         |     1 +
/dev/null => jq-1.6/modules/oniguruma/.gitignore   |    27 +
/dev/null => jq-1.6/modules/oniguruma/AUTHORS      |     1 +
.../modules/oniguruma/CMakeLists.txt               |    86 +
/dev/null => jq-1.6/modules/oniguruma/COPYING      |    28 +
/dev/null => jq-1.6/modules/oniguruma/ChangeLog    |     0
/dev/null => jq-1.6/modules/oniguruma/HISTORY      |  2138 ++
/dev/null => jq-1.6/modules/oniguruma/INSTALL      |   370 +
/dev/null => jq-1.6/modules/oniguruma/Makefile.am  |    33 +
/dev/null => jq-1.6/modules/oniguruma/NEWS         |     0
/dev/null => jq-1.6/modules/oniguruma/README       |   189 +
/dev/null => jq-1.6/modules/oniguruma/README.ja    |   195 +
/dev/null => jq-1.6/modules/oniguruma/README.md    |   203 +
.../modules/oniguruma/cmake/dist.cmake             |   321 +
--- cut here ---

... but this took no effect on OpenWrt as oniguruma was explicitly disabled.

However I would suggest updating the PKG_HASH in OpenWrt Makefile & removing the old archive from mirror host to keep it constant with upstream.

[neheb]

Given that upstream has done this, I think it makes more sense to switch to PKG_SOURCE_PROTO:=git

[1715173329]

Given that upstream has done this, I think it makes more sense to switch to PKG_SOURCE_PROTO:=git

+1

[1715173329]

hummmm... need autoreconf?

[bradfordzhang]

I can't.... I'm so noob

On 11月 17 2022, at 5:02 下午, Tianling Shen @.***> wrote:

hummmm... need autoreconf? — Reply to this email directly, view it on GitHub (https://github.com/openwrt/packages/pull/19496#issuecomment-1318305094), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AMJ274J5HJ6FVW5YCE6AHBLWIXYBJANCNFSM6AAAAAAQZZ3B44). You are receiving this because you were mentioned.