libpsl with icu tests fail on Alpine Linux

[neheb]

./psl/subprojects/libpsl-0.21.2/tests/test-registrable-domain
have 9458 suffixes and 8 exceptions
psl_str_to_utf8lower(www.�yer.no)=-2
psl_str_to_utf8lower(�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)=-2
psl_str_to_utf8lower(�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)=-2
psl_str_to_utf8lower(�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)=-2
psl_str_to_utf8lower(�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)=-2

https://gist.github.com/neheb/923f3acf8a60d262c794c60d9a05e53b

[rockdaboot]

Can you provide config.log? What is the output of locale? What ICU version are you using? What compiler/version are you using? Does the same happen with libidn2?

[neheb]

I’ll get that later today.

libidn2 or 1 are not impacted. Just ICU.

Alpine uses musl. That’s the differing factor.

[neheb]

mangix@alpine ~/d/wrapdb (master) [2]> locale fish: Unknown command: locale

https://gist.github.com/neheb/6c8a6ec3c378714788fdd4eec95d94ca

icu is 73.2. gcc is 12.2.1

[neheb]

testing again with alpine's own icu, test-is-public-all succeeds but test-registrable-domain still fails. Same output. ping @amyspark

[rockdaboot]

Hm, -2 means

PSL_ERR_CONVERTER = -2, /* failed to open libicu utf-16 converter */

Maybe your libicu is built without utf-16 support!?

[kloczek]

I have similar issue but with icu 74.1 (with 73.2 was OK)

+ /usr/bin/meson test -C x86_64-redhat-linux-gnu --num-processes 48 --print-errorlogs
ninja: Entering directory `/home/tkloczko/rpmbuild/BUILD/libpsl-0.21.2/x86_64-redhat-linux-gnu'
ninja: no work to do.
1/6 test-is-public                   OK              0.04s
2/6 test-is-cookie-domain-acceptable OK              0.03s
3/6 libpsl_idn2_fuzzer               OK              0.02s
4/6 libpsl_idn2_load_dafsa_fuzzer    OK              0.01s
5/6 test-is-public-all               FAIL            0.11s   exit status 1
>>> MALLOC_PERTURB_=173 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 LD_LIBRARY_PATH=/home/tkloczko/rpmbuild/BUILD/libpsl-0.21.2/x86_64-redhat-linux-gnu/src /home/tkloczko/rpmbuild/BUILD/libpsl-0.21.2/x86_64-redhat-linux-gnu/tests/test-is-public-all
――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
loaded 9559 suffixes and 8 exceptions
builtin PSL has -1 suffixes and -1 exceptions
psl_is_public_suffix2(desi, PSL_TYPE_ICANN|PSL_TYPE_NO_STAR_RULE)=0 (expected 1)
psl_is_public_suffix2(desi, PSL_TYPE_ANY|PSL_TYPE_NO_STAR_RULE)=0 (expected 1)
psl_is_public_suffix2(wed, PSL_TYPE_ICANN|PSL_TYPE_NO_STAR_RULE)=0 (expected 1)
psl_is_public_suffix2(wed, PSL_TYPE_ANY|PSL_TYPE_NO_STAR_RULE)=0 (expected 1)
psl_is_public_suffix2(torun.pl, PSL_TYPE_PRIVATE)=0 (expected 1)
psl_is_public_suffix2(torun.pl, PSL_TYPE_PRIVATE|PSL_TYPE_NO_STAR_RULE)=0 (expected 1)
psl_is_public_suffix2(torun.pl, PSL_TYPE_ANY)=0 (expected 1)
psl_is_public_suffix2(torun.pl, PSL_TYPE_ANY|PSL_TYPE_NO_STAR_RULE)=0 (expected 1)
Summary: 8 out of 235848 tests failed
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

6/6 libpsl_idn2_load_fuzzer          OK              0.18s

Summary of Failures:

5/6 test-is-public-all               FAIL            0.11s   exit status 1

Ok:                 5
Expected Fail:      0
Fail:               1
Unexpected Pass:    0
Skipped:            0
Timeout:            0

[rockdaboot]

I tested with libicu 74.2-1 from Debian experimental and I don't see any failing tests. ASAN and UBSAN where both enabled.

Anyway, this doesn't seem to be an issue with libpsl, more with certain builds of libicu.