node icon indicating copy to clipboard operation
node copied to clipboard
verified publisher icon nodejs

deps: update OpenSSL to 3.5

Open targos opened this issue 8 months ago • 28 comments

Refs: https://github.com/openssl/openssl/releases/tag/openssl-3.5.0

targos avatar Apr 08 '25 16:04 targos

Review requested:

  • [ ] @nodejs/security-wg

nodejs-github-bot avatar Apr 08 '25 16:04 nodejs-github-bot

Beat me to it 😆

ranisalt avatar Apr 08 '25 16:04 ranisalt

I tried to run it locally. The download step seems to work but I don't have the dependencies to try the regenerate one.

targos avatar Apr 08 '25 16:04 targos

@ranisalt Feel free to reopen your PR and I'll close this one. Don't want to steal your first contribution!

targos avatar Apr 08 '25 16:04 targos

It's not that simple, unfortunately. It will replace the tree inside deps/openssl but will fail to build.

I'm currently picking apart how we converted the OpenSSL 3.0 build to gyp to work out what needs to be updated (ideally without hardcoding filenames everywhere).

richardlau avatar Apr 08 '25 16:04 richardlau

I ran the update script on my fork to generate the commits so we can see what issues need fixing.

targos avatar Apr 09 '25 05:04 targos

Locally I get this first error:

FAILED: obj/deps/openssl/openssl/ssl/openssl.ssl_ciph.o 
cc -MMD -MF obj/deps/openssl/openssl/ssl/openssl.ssl_ciph.o.d -D_GLIBCXX_USE_CXX11_ABI=1 -DNODE_OPENSSL_CONF_NAME=nodejs_conf -DICU_NO_USER_DATA_OVERRIDE -D_DARWIN_USE_64_BIT_INODE=1 -DOPENSSL_NO_PINSHARED -DOPENSSL_THREADS -DOPENSSL_NO_HW -DOPENSSL_API_COMPAT=0x10100001L -DSTATIC_LEGACY -DNDEBUG -DL_ENDIAN -DOPENSSL_BUILDING_OPENSSL -DBSAES_ASM -DECP_NISTZ256_ASM -DECP_SM2P256_ASM -DKECCAK1600_ASM -DMD5_ASM -DOPENSSL_BN_ASM_MONT -DOPENSSL_CPUID_OBJ -DOPENSSL_SM3_ASM -DPOLY1305_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DSM4_ASM -DVPAES_ASM -DVPSM4_ASM -DOPENSSL_PIC '-DMODULESDIR="/Users/mzasso/git/nodejs/node/out/out/Debug/obj.target/deps/openssl/lib/openssl-modules"' '-DOPENSSLDIR="/System/Library/OpenSSL/"' '-DENGINESDIR="/dev/null"' -I../../deps/openssl/openssl -I../../deps/openssl/openssl/include -I../../deps/openssl/openssl/crypto -I../../deps/openssl/openssl/crypto/include -I../../deps/openssl/openssl/crypto/modes -I../../deps/openssl/openssl/crypto/ec/curve448 -I../../deps/openssl/openssl/crypto/ec/curve448/arch_32 -I../../deps/openssl/openssl/providers/common/include -I../../deps/openssl/openssl/providers/implementations/include -I../../deps/openssl/config -I../../deps/openssl/config/archs/darwin64-arm64-cc/asm_avx2 -I../../deps/openssl/config/archs/darwin64-arm64-cc/asm_avx2/include -I../../deps/openssl/config/archs/darwin64-arm64-cc/asm_avx2/crypto -I../../deps/openssl/config/archs/darwin64-arm64-cc/asm_avx2/crypto/include/internal -I../../deps/openssl/config/archs/darwin64-arm64-cc/asm_avx2/providers/common/include -O3 -gdwarf-2 -fno-strict-aliasing -mmacosx-version-min=13.5 -arch arm64 -Wall -Wendif-labels -W -Wno-unused-parameter -Wno-missing-field-initializers   -c ../../deps/openssl/openssl/ssl/ssl_ciph.c -o obj/deps/openssl/openssl/ssl/openssl.ssl_ciph.o
../../deps/openssl/openssl/ssl/ssl_ciph.c:15:10: fatal error: 'openssl/comp.h' file not found
   15 | #include <openssl/comp.h>
      |          ^~~~~~~~~~~~~~~~
1 error generated.

targos avatar Apr 09 '25 05:04 targos

OK, I think comp.h is not generated because we explicitly pass no-comp: https://github.com/targos/node/blob/a0a0297126347145784e793ea057892c47763269/deps/openssl/config/Makefile#L24-L28

targos avatar Apr 10 '25 08:04 targos

That was introduced in https://github.com/nodejs/node/pull/19794 with the comment "against CRIME attack" in the code.

targos avatar Apr 10 '25 08:04 targos

On main, we have comp.h: https://github.com/nodejs/node/blob/9bbbe60f6b0001964cb62182ce17e7f3980ccf06/deps/openssl/openssl/include/openssl/comp.h

targos avatar Apr 10 '25 08:04 targos

In OpenSSL 3.5 comp.h is now generated from comp.h.in.

Before the CI security incident kicked off, I'd started looking at how we translated the OpenSSL build to Node.js. Ref: https://github.com/nodejs/node/blob/main/deps/openssl/README.md

The second part of the update script runs a configuration step that runs through OpenSSL's Perl scripts to generate additional files.

I've tried this diff to add generation of comp.h during the second part of the update process (note the Makefiles say not to edit, but our own deps/openssl/README.md says these are hand crafted):

diff --git a/deps/openssl/config/Makefile b/deps/openssl/config/Makefile
index 48d2af80019..c8e6889fb70 100644
--- a/deps/openssl/config/Makefile
+++ b/deps/openssl/config/Makefile
@@ -37,7 +37,7 @@ OPSSL_SRC = ../openssl
 # Header files generated with Configure
 #INT_CFGS = bn_conf.h dso_conf.h
 INT_CFG_DIR = $(OPSSL_SRC)/include/crypto
-GEN_HEADERS = asn1 asn1t bio cmp cms configuration conf crmf crypto ct err    \
+GEN_HEADERS = asn1 asn1t bio comp cmp cms configuration conf crmf crypto ct err    \
               ess fipskey lhash ocsp opensslv pkcs12 pkcs7 safestack srp ssl  \
              ui x509 x509v3 x509_vfy conf

diff --git a/deps/openssl/config/Makefile_VC-WIN32 b/deps/openssl/config/Makefile_VC-WIN32
index fdbef72361b..b5c746fb2df 100644
--- a/deps/openssl/config/Makefile_VC-WIN32
+++ b/deps/openssl/config/Makefile_VC-WIN32
@@ -1,7 +1,7 @@
 BLDDIR=.
 PERL=perl
 RM= rm -f
-GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
+GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/comp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
 GENERATED=apps/CA.pl apps/openssl.rc apps/progs.h apps/tsget.pl crypto/aes/aes-586.asm crypto/aes/aesni-x86.asm crypto/aes/vpaes-x86.asm crypto/bf/bf-586.asm crypto/bn/bn-586.asm crypto/bn/co-586.asm crypto/bn/x86-gf2m.asm crypto/bn/x86-mont.asm crypto/buildinf.h crypto/camellia/cmll-x86.asm crypto/chacha/chacha-x86.asm crypto/des/crypt586.asm crypto/des/des-586.asm crypto/ec/ecp_nistz256-x86.asm crypto/md5/md5-586.asm crypto/modes/ghash-x86.asm crypto/poly1305/poly1305-x86.asm crypto/rc4/rc4-586.asm crypto/ripemd/rmd-586.asm crypto/sha/sha1-586.asm crypto/sha/sha256-586.asm crypto/sha/sha512-586.asm crypto/whrlpool/wp-mmx.asm crypto/x86cpuid.asm engines/capi.def engines/dasync.def engines/e_padlock-x86.asm engines/ossltest.def engines/padlock.def libcrypto.def libcrypto.rc libssl.def libssl.rc test/buildtest_aes.c test/buildtest_asn1.c test/buildtest_asn1t.c test/buildtest_async.c test/buildtest_bio.c test/buildtest_blowfish.c test/buildtest_bn.c test/buildtest_buffer.c test/buildtest_camellia.c test/buildtest_cast.c test/buildtest_cmac.c test/buildtest_cms.c test/buildtest_comp.c test/buildtest_conf.c test/buildtest_conf_api.c test/buildtest_crypto.c test/buildtest_ct.c test/buildtest_des.c test/buildtest_dh.c test/buildtest_dsa.c test/buildtest_dtls1.c test/buildtest_e_os2.c test/buildtest_ebcdic.c test/buildtest_ec.c test/buildtest_ecdh.c test/buildtest_ecdsa.c test/buildtest_engine.c test/buildtest_evp.c test/buildtest_hmac.c test/buildtest_idea.c test/buildtest_kdf.c test/buildtest_lhash.c test/buildtest_md4.c test/buildtest_md5.c test/buildtest_mdc2.c test/buildtest_modes.c test/buildtest_obj_mac.c test/buildtest_objects.c test/buildtest_ocsp.c test/buildtest_opensslv.c test/buildtest_ossl_typ.c test/buildtest_pem.c test/buildtest_pem2.c test/buildtest_pkcs12.c test/buildtest_pkcs7.c test/buildtest_rand.c test/buildtest_rand_drbg.c test/buildtest_rc2.c test/buildtest_rc4.c test/buildtest_ripemd.c test/buildtest_rsa.c test/buildtest_safestack.c test/buildtest_seed.c test/buildtest_sha.c test/buildtest_srp.c test/buildtest_srtp.c test/buildtest_ssl.c test/buildtest_ssl2.c test/buildtest_stack.c test/buildtest_store.c test/buildtest_symhacks.c test/buildtest_tls1.c test/buildtest_ts.c test/buildtest_txt_db.c test/buildtest_ui.c test/buildtest_whrlpool.c test/buildtest_x509.c test/buildtest_x509_vfy.c test/buildtest_x509v3.c tools/c_rehash.pl providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c

 # Variables starting with LIB_ are used to build library object files
@@ -54,6 +54,9 @@ include/openssl/bio.h: include/openssl/bio.h.in configdata.pm
 include/openssl/cmp.h: include/openssl/cmp.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cmp.h.in" > $@
+include/openssl/comp.h: include/openssl/comp.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/comp.h.in" > $@
 include/openssl/cms.h: include/openssl/cms.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cms.h.in" > $@
diff --git a/deps/openssl/config/Makefile_VC-WIN64-ARM b/deps/openssl/config/Makefile_VC-WIN64-ARM
index 52fc9cd9cf4..4a1d558a2a0 100644
--- a/deps/openssl/config/Makefile_VC-WIN64-ARM
+++ b/deps/openssl/config/Makefile_VC-WIN64-ARM
@@ -16,7 +16,7 @@ MINOR=1.1

 SHLIB_VERSION_NUMBER=1.1

-GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
+GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/comp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
 GENERATED=crypto/buildinf.h apps/progs.h providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c apps/progs.c providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c

 INSTALL_LIBS="libcrypto.lib" "libssl.lib"
@@ -157,6 +157,9 @@ include/openssl/bio.h: include/openssl/bio.h.in configdata.pm
 include/openssl/cmp.h: include/openssl/cmp.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cmp.h.in" > $@
+include/openssl/comp.h: include/openssl/comp.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/comp.h.in" > $@
 include/openssl/cms.h: include/openssl/cms.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cms.h.in" > $@
diff --git a/deps/openssl/config/Makefile_VC-WIN64A b/deps/openssl/config/Makefile_VC-WIN64A
index d8fde1e0f39..c9ac8a33a15 100644
--- a/deps/openssl/config/Makefile_VC-WIN64A
+++ b/deps/openssl/config/Makefile_VC-WIN64A
@@ -5,7 +5,7 @@ RM= rm -f
 AS=nasm
 ASFLAGS=-g

-GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
+GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/comp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
 GENERATED=apps/CA.pl apps/openssl.rc apps/progs.h apps/tsget.pl crypto/aes/aes-x86_64.asm crypto/aes/aesni-mb-x86_64.asm crypto/aes/aesni-sha1-x86_64.asm crypto/aes/aesni-sha256-x86_64.asm crypto/aes/aesni-x86_64.asm crypto/aes/bsaes-x86_64.asm crypto/aes/vpaes-x86_64.asm crypto/bn/rsaz-avx2.asm crypto/bn/rsaz-x86_64.asm crypto/bn/x86_64-gf2m.asm crypto/bn/x86_64-mont.asm crypto/bn/x86_64-mont5.asm crypto/buildinf.h crypto/camellia/cmll-x86_64.asm crypto/chacha/chacha-x86_64.asm crypto/ec/ecp_nistz256-x86_64.asm crypto/ec/x25519-x86_64.asm crypto/md5/md5-x86_64.asm crypto/modes/aesni-gcm-x86_64.asm crypto/modes/ghash-x86_64.asm crypto/poly1305/poly1305-x86_64.asm crypto/rc4/rc4-md5-x86_64.asm crypto/rc4/rc4-x86_64.asm crypto/sha/keccak1600-x86_64.asm crypto/sha/sha1-mb-x86_64.asm crypto/sha/sha1-x86_64.asm crypto/sha/sha256-mb-x86_64.asm crypto/sha/sha256-x86_64.asm crypto/sha/sha512-x86_64.asm crypto/uplink-x86_64.asm crypto/whrlpool/wp-x86_64.asm crypto/x86_64cpuid.asm engines/e_padlock-x86_64.asm libcrypto.def libcrypto.rc libssl.def libssl.rc test/buildtest_aes.c test/buildtest_asn1.c test/buildtest_asn1t.c test/buildtest_async.c test/buildtest_bio.c test/buildtest_blowfish.c test/buildtest_bn.c test/buildtest_buffer.c test/buildtest_camellia.c test/buildtest_cast.c test/buildtest_cmac.c test/buildtest_cms.c test/buildtest_comp.c test/buildtest_conf.c test/buildtest_conf_api.c test/buildtest_crypto.c test/buildtest_ct.c test/buildtest_des.c test/buildtest_dh.c test/buildtest_dsa.c test/buildtest_dtls1.c test/buildtest_e_os2.c test/buildtest_ebcdic.c test/buildtest_ec.c test/buildtest_ecdh.c test/buildtest_ecdsa.c test/buildtest_engine.c test/buildtest_evp.c test/buildtest_hmac.c test/buildtest_idea.c test/buildtest_kdf.c test/buildtest_lhash.c test/buildtest_md4.c test/buildtest_md5.c test/buildtest_mdc2.c test/buildtest_modes.c test/buildtest_obj_mac.c test/buildtest_objects.c test/buildtest_ocsp.c test/buildtest_opensslv.c test/buildtest_ossl_typ.c test/buildtest_pem.c test/buildtest_pem2.c test/buildtest_pkcs12.c test/buildtest_pkcs7.c test/buildtest_rand.c test/buildtest_rand_drbg.c test/buildtest_rc2.c test/buildtest_rc4.c test/buildtest_ripemd.c test/buildtest_rsa.c test/buildtest_safestack.c test/buildtest_seed.c test/buildtest_sha.c test/buildtest_srp.c test/buildtest_srtp.c test/buildtest_ssl.c test/buildtest_ssl2.c test/buildtest_stack.c test/buildtest_store.c test/buildtest_symhacks.c test/buildtest_tls1.c test/buildtest_ts.c test/buildtest_txt_db.c test/buildtest_ui.c test/buildtest_whrlpool.c test/buildtest_x509.c test/buildtest_x509_vfy.c test/buildtest_x509v3.c tools/c_rehash.pl providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c

 PERLASM_SCHEME= auto
@@ -30,6 +30,9 @@ include/openssl/bio.h: include/openssl/bio.h.in configdata.pm
 include/openssl/cmp.h: include/openssl/cmp.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cmp.h.in" > $@
+include/openssl/comp.h: include/openssl/comp.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/comp.h.in" > $@
 include/openssl/cms.h: include/openssl/cms.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cms.h.in" > $@

but this now errors on the next now generated but wasn't before header:

../deps/openssl/openssl/ssl/t1_lib.c:16:10: fatal error: openssl/core_names.h: No such file or directory
   16 | #include <openssl/core_names.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Comparing all of the *.in files between OpenSSL 3.0 (from a v20.x-staging checkout) and 3.5:

--- ../openssl30.in.sort.txt    2025-04-10 12:27:12.373225504 +0000
+++ ../openssl35.in.sort.txt    2025-04-10 12:27:23.287465350 +0000
@@ -1,6 +1,7 @@
 deps/openssl/openssl/apps/CA.pl.in
 deps/openssl/openssl/apps/tsget.in
 deps/openssl/openssl/configdata.pm.in
+deps/openssl/openssl/crypto/params_idx.c.in
 deps/openssl/openssl/doc/build.info.in
 deps/openssl/openssl/doc/man1/openssl-asn1parse.pod.in
 deps/openssl/openssl/doc/man1/openssl-ca.pod.in
@@ -44,6 +45,7 @@
 deps/openssl/openssl/doc/man1/openssl-rsautl.pod.in
 deps/openssl/openssl/doc/man1/openssl-s_client.pod.in
 deps/openssl/openssl/doc/man1/openssl-sess_id.pod.in
+deps/openssl/openssl/doc/man1/openssl-skeyutl.pod.in
 deps/openssl/openssl/doc/man1/openssl-smime.pod.in
 deps/openssl/openssl/doc/man1/openssl-speed.pod.in
 deps/openssl/openssl/doc/man1/openssl-spkac.pod.in
@@ -56,15 +58,23 @@
 deps/openssl/openssl/doc/man1/openssl-version.pod.in
 deps/openssl/openssl/doc/man1/openssl-x509.pod.in
 deps/openssl/openssl/doc/man7/openssl_user_macros.pod.in
+deps/openssl/openssl/exporters/cmake/OpenSSLConfig.cmake.in
+deps/openssl/openssl/exporters/cmake/OpenSSLConfigVersion.cmake.in
+deps/openssl/openssl/exporters/pkg-config/libcrypto.pc.in
+deps/openssl/openssl/exporters/pkg-config/libssl.pc.in
+deps/openssl/openssl/exporters/pkg-config/openssl.pc.in
 deps/openssl/openssl/include/crypto/bn_conf.h.in
 deps/openssl/openssl/include/crypto/dso_conf.h.in
+deps/openssl/openssl/include/internal/param_names.h.in
 deps/openssl/openssl/include/openssl/asn1.h.in
 deps/openssl/openssl/include/openssl/asn1t.h.in
 deps/openssl/openssl/include/openssl/bio.h.in
 deps/openssl/openssl/include/openssl/cmp.h.in
 deps/openssl/openssl/include/openssl/cms.h.in
+deps/openssl/openssl/include/openssl/comp.h.in
 deps/openssl/openssl/include/openssl/conf.h.in
 deps/openssl/openssl/include/openssl/configuration.h.in
+deps/openssl/openssl/include/openssl/core_names.h.in
 deps/openssl/openssl/include/openssl/crmf.h.in
 deps/openssl/openssl/include/openssl/crypto.h.in
 deps/openssl/openssl/include/openssl/ct.h.in
@@ -80,26 +90,34 @@
 deps/openssl/openssl/include/openssl/srp.h.in
 deps/openssl/openssl/include/openssl/ssl.h.in
 deps/openssl/openssl/include/openssl/ui.h.in
+deps/openssl/openssl/include/openssl/x509_acert.h.in
 deps/openssl/openssl/include/openssl/x509.h.in
 deps/openssl/openssl/include/openssl/x509v3.h.in
 deps/openssl/openssl/include/openssl/x509_vfy.h.in
+deps/openssl/openssl/makefile.in
+deps/openssl/openssl/Makefile.in
 deps/openssl/openssl/providers/common/der/der_digests_gen.c.in
 deps/openssl/openssl/providers/common/der/der_dsa_gen.c.in
 deps/openssl/openssl/providers/common/der/der_ec_gen.c.in
 deps/openssl/openssl/providers/common/der/der_ecx_gen.c.in
+deps/openssl/openssl/providers/common/der/der_ml_dsa_gen.c.in
 deps/openssl/openssl/providers/common/der/der_rsa_gen.c.in
+deps/openssl/openssl/providers/common/der/der_slh_dsa_gen.c.in
 deps/openssl/openssl/providers/common/der/der_sm2_gen.c.in
 deps/openssl/openssl/providers/common/der/der_wrap_gen.c.in
 deps/openssl/openssl/providers/common/include/prov/der_digests.h.in
 deps/openssl/openssl/providers/common/include/prov/der_dsa.h.in
 deps/openssl/openssl/providers/common/include/prov/der_ec.h.in
 deps/openssl/openssl/providers/common/include/prov/der_ecx.h.in
+deps/openssl/openssl/providers/common/include/prov/der_ml_dsa.h.in
 deps/openssl/openssl/providers/common/include/prov/der_rsa.h.in
+deps/openssl/openssl/providers/common/include/prov/der_slh_dsa.h.in
 deps/openssl/openssl/providers/common/include/prov/der_sm2.h.in
 deps/openssl/openssl/providers/common/include/prov/der_wrap.h.in
 deps/openssl/openssl/test/dane-cross.in
 deps/openssl/openssl/test/danetest.in
 deps/openssl/openssl/test/provider_internal_test.cnf.in
+deps/openssl/openssl/test/recipes/95-test_external_tlsfuzzer_data/cert.json.in
 deps/openssl/openssl/test/ssl-tests/01-simple.cnf.in
 deps/openssl/openssl/test/ssl-tests/02-protocol-version.cnf.in
 deps/openssl/openssl/test/ssl-tests/03-custom_verify.cnf.in
@@ -130,10 +148,9 @@
 deps/openssl/openssl/test/ssl-tests/28-seclevel.cnf.in
 deps/openssl/openssl/test/ssl-tests/29-dtls-sctp-label-bug.cnf.in
 deps/openssl/openssl/test/ssl-tests/30-extended-master-secret.cnf.in
+deps/openssl/openssl/test/ssl-tests/32-compressed-certificate.cnf.in
 deps/openssl/openssl/tools/c_rehash.in
-deps/openssl/openssl/util/local_shlib.com.in
 deps/openssl/openssl/util/shlib_wrap.sh.in
-deps/openssl/openssl/util/unlocal_shlib.com.in
 deps/openssl/openssl/util/wrap.pl.in
 deps/openssl/openssl/VMS/openssl_ivp.com.in
 deps/openssl/openssl/VMS/openssl_shutdown.com.in

richardlau avatar Apr 10 '25 12:04 richardlau

That's one big PR 🤯

khaosdoctor avatar Apr 10 '25 17:04 khaosdoctor

image

Ummmm..... good lord. It's going to be quite difficult to review this, unfortunately. The GitHub UI hangs and crashes whenever I try to open the changes tab.

jasnell avatar Apr 10 '25 18:04 jasnell

Unfortunately I've run out of time before being out until after Easter. At least for the next few days I'm unlikely to be online.

This is as far as I've got (adding core_names.h and x509_acert.h):

diff --git a/deps/openssl/config/Makefile b/deps/openssl/config/Makefile
index 48d2af80019..e1ad0bb8e90 100644
--- a/deps/openssl/config/Makefile
+++ b/deps/openssl/config/Makefile
@@ -37,9 +37,9 @@ OPSSL_SRC = ../openssl
 # Header files generated with Configure
 #INT_CFGS = bn_conf.h dso_conf.h
 INT_CFG_DIR = $(OPSSL_SRC)/include/crypto
-GEN_HEADERS = asn1 asn1t bio cmp cms configuration conf crmf crypto ct err    \
-              ess fipskey lhash ocsp opensslv pkcs12 pkcs7 safestack srp ssl  \
-             ui x509 x509v3 x509_vfy conf
+GEN_HEADERS = asn1 asn1t bio comp cmp cms conf configuration core_names crmf \
+              crypto ct err ess fipskey lhash ocsp opensslv pkcs12 pkcs7 \
+              safestack srp ssl ui x509_acert x509 x509v3 x509_vfy

 CRYPTO_GEN_HEADERS = bn_conf dso_conf

diff --git a/deps/openssl/config/Makefile_VC-WIN32 b/deps/openssl/config/Makefile_VC-WIN32
index fdbef72361b..7584152ac3f 100644
--- a/deps/openssl/config/Makefile_VC-WIN32
+++ b/deps/openssl/config/Makefile_VC-WIN32
@@ -1,7 +1,7 @@
 BLDDIR=.
 PERL=perl
 RM= rm -f
-GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
+GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/comp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/core_names.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_acert.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
 GENERATED=apps/CA.pl apps/openssl.rc apps/progs.h apps/tsget.pl crypto/aes/aes-586.asm crypto/aes/aesni-x86.asm crypto/aes/vpaes-x86.asm crypto/bf/bf-586.asm crypto/bn/bn-586.asm crypto/bn/co-586.asm crypto/bn/x86-gf2m.asm crypto/bn/x86-mont.asm crypto/buildinf.h crypto/camellia/cmll-x86.asm crypto/chacha/chacha-x86.asm crypto/des/crypt586.asm crypto/des/des-586.asm crypto/ec/ecp_nistz256-x86.asm crypto/md5/md5-586.asm crypto/modes/ghash-x86.asm crypto/poly1305/poly1305-x86.asm crypto/rc4/rc4-586.asm crypto/ripemd/rmd-586.asm crypto/sha/sha1-586.asm crypto/sha/sha256-586.asm crypto/sha/sha512-586.asm crypto/whrlpool/wp-mmx.asm crypto/x86cpuid.asm engines/capi.def engines/dasync.def engines/e_padlock-x86.asm engines/ossltest.def engines/padlock.def libcrypto.def libcrypto.rc libssl.def libssl.rc test/buildtest_aes.c test/buildtest_asn1.c test/buildtest_asn1t.c test/buildtest_async.c test/buildtest_bio.c test/buildtest_blowfish.c test/buildtest_bn.c test/buildtest_buffer.c test/buildtest_camellia.c test/buildtest_cast.c test/buildtest_cmac.c test/buildtest_cms.c test/buildtest_comp.c test/buildtest_conf.c test/buildtest_conf_api.c test/buildtest_crypto.c test/buildtest_ct.c test/buildtest_des.c test/buildtest_dh.c test/buildtest_dsa.c test/buildtest_dtls1.c test/buildtest_e_os2.c test/buildtest_ebcdic.c test/buildtest_ec.c test/buildtest_ecdh.c test/buildtest_ecdsa.c test/buildtest_engine.c test/buildtest_evp.c test/buildtest_hmac.c test/buildtest_idea.c test/buildtest_kdf.c test/buildtest_lhash.c test/buildtest_md4.c test/buildtest_md5.c test/buildtest_mdc2.c test/buildtest_modes.c test/buildtest_obj_mac.c test/buildtest_objects.c test/buildtest_ocsp.c test/buildtest_opensslv.c test/buildtest_ossl_typ.c test/buildtest_pem.c test/buildtest_pem2.c test/buildtest_pkcs12.c test/buildtest_pkcs7.c test/buildtest_rand.c test/buildtest_rand_drbg.c test/buildtest_rc2.c test/buildtest_rc4.c test/buildtest_ripemd.c test/buildtest_rsa.c test/buildtest_safestack.c test/buildtest_seed.c test/buildtest_sha.c test/buildtest_srp.c test/buildtest_srtp.c test/buildtest_ssl.c test/buildtest_ssl2.c test/buildtest_stack.c test/buildtest_store.c test/buildtest_symhacks.c test/buildtest_tls1.c test/buildtest_ts.c test/buildtest_txt_db.c test/buildtest_ui.c test/buildtest_whrlpool.c test/buildtest_x509.c test/buildtest_x509_vfy.c test/buildtest_x509v3.c tools/c_rehash.pl providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c

 # Variables starting with LIB_ are used to build library object files
@@ -54,6 +54,9 @@ include/openssl/bio.h: include/openssl/bio.h.in configdata.pm
 include/openssl/cmp.h: include/openssl/cmp.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cmp.h.in" > $@
+include/openssl/comp.h: include/openssl/comp.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/comp.h.in" > $@
 include/openssl/cms.h: include/openssl/cms.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cms.h.in" > $@
@@ -66,6 +69,9 @@ include/openssl/conf.h: include/openssl/conf.h.in configdata.pm
 include/openssl/configuration.h: include/openssl/configuration.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/configuration.h.in" > $@
+include/openssl/core_names.h: include/openssl/core_names.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/core_names.h.in" > $@
 include/openssl/crmf.h: include/openssl/crmf.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/crmf.h.in" > $@
@@ -117,6 +123,9 @@ include/openssl/ui.h: include/openssl/ui.h.in configdata.pm
 include/openssl/x509.h: include/openssl/x509.h.in configdata.pm
        $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
            "-omakefile" "include/openssl/x509.h.in" > $@
+include/openssl/x509_acert.h: include/openssl/x509_acert.h.in configdata.pm
+       $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
+           "-omakefile" "include/openssl/x509_acert.h.in" > $@
 include/openssl/x509_vfy.h: include/openssl/x509_vfy.h.in configdata.pm
        $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
            "-omakefile" "include/openssl/x509_vfy.h.in" > $@
diff --git a/deps/openssl/config/Makefile_VC-WIN64-ARM b/deps/openssl/config/Makefile_VC-WIN64-ARM
index 52fc9cd9cf4..edc52c8ccf3 100644
--- a/deps/openssl/config/Makefile_VC-WIN64-ARM
+++ b/deps/openssl/config/Makefile_VC-WIN64-ARM
@@ -16,7 +16,7 @@ MINOR=1.1

 SHLIB_VERSION_NUMBER=1.1

-GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
+GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/comp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/core_names.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_acert.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
 GENERATED=crypto/buildinf.h apps/progs.h providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c apps/progs.c providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c

 INSTALL_LIBS="libcrypto.lib" "libssl.lib"
@@ -157,6 +157,9 @@ include/openssl/bio.h: include/openssl/bio.h.in configdata.pm
 include/openssl/cmp.h: include/openssl/cmp.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cmp.h.in" > $@
+include/openssl/comp.h: include/openssl/comp.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/comp.h.in" > $@
 include/openssl/cms.h: include/openssl/cms.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cms.h.in" > $@
@@ -169,6 +172,9 @@ include/openssl/conf.h: include/openssl/conf.h.in configdata.pm
 include/openssl/configuration.h: include/openssl/configuration.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/configuration.h.in" > $@
+include/openssl/core_names.h: include/openssl/core_names.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/core_names.h.in" > $@
 include/openssl/crmf.h: include/openssl/crmf.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/crmf.h.in" > $@
@@ -220,6 +226,9 @@ include/openssl/ui.h: include/openssl/ui.h.in configdata.pm
 include/openssl/x509.h: include/openssl/x509.h.in configdata.pm
        $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
            "-omakefile" "include/openssl/x509.h.in" > $@
+include/openssl/x509_acert.h: include/openssl/x509_acert.h.in configdata.pm
+       $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
+           "-omakefile" "include/openssl/x509_acert.h.in" > $@
 include/openssl/x509_vfy.h: include/openssl/x509_vfy.h.in configdata.pm
        $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
            "-omakefile" "include/openssl/x509_vfy.h.in" > $@
diff --git a/deps/openssl/config/Makefile_VC-WIN64A b/deps/openssl/config/Makefile_VC-WIN64A
index d8fde1e0f39..c5e2a83ac00 100644
--- a/deps/openssl/config/Makefile_VC-WIN64A
+++ b/deps/openssl/config/Makefile_VC-WIN64A
@@ -5,7 +5,7 @@ RM= rm -f
 AS=nasm
 ASFLAGS=-g

-GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
+GENERATED_MANDATORY=include/crypto/bn_conf.h include/crypto/dso_conf.h include/openssl/asn1.h include/openssl/asn1t.h include/openssl/bio.h include/openssl/cmp.h include/openssl/comp.h include/openssl/cms.h include/openssl/conf.h include/openssl/configuration.h include/openssl/core_names.h include/openssl/crmf.h include/openssl/crypto.h include/openssl/ct.h include/openssl/err.h include/openssl/ess.h include/openssl/fipskey.h include/openssl/lhash.h include/openssl/ocsp.h include/openssl/opensslv.h include/openssl/pkcs12.h include/openssl/pkcs7.h include/openssl/safestack.h include/openssl/srp.h include/openssl/ssl.h include/openssl/ui.h include/openssl/x509.h include/openssl/x509_acert.h include/openssl/x509_vfy.h include/openssl/x509v3.h providers/common/include/prov/der_digests.h providers/common/include/prov/der_dsa.h providers/common/include/prov/der_ec.h providers/common/include/prov/der_ecx.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_rsa.h providers/common/include/prov/der_sm2.h providers/common/include/prov/der_wrap.h
 GENERATED=apps/CA.pl apps/openssl.rc apps/progs.h apps/tsget.pl crypto/aes/aes-x86_64.asm crypto/aes/aesni-mb-x86_64.asm crypto/aes/aesni-sha1-x86_64.asm crypto/aes/aesni-sha256-x86_64.asm crypto/aes/aesni-x86_64.asm crypto/aes/bsaes-x86_64.asm crypto/aes/vpaes-x86_64.asm crypto/bn/rsaz-avx2.asm crypto/bn/rsaz-x86_64.asm crypto/bn/x86_64-gf2m.asm crypto/bn/x86_64-mont.asm crypto/bn/x86_64-mont5.asm crypto/buildinf.h crypto/camellia/cmll-x86_64.asm crypto/chacha/chacha-x86_64.asm crypto/ec/ecp_nistz256-x86_64.asm crypto/ec/x25519-x86_64.asm crypto/md5/md5-x86_64.asm crypto/modes/aesni-gcm-x86_64.asm crypto/modes/ghash-x86_64.asm crypto/poly1305/poly1305-x86_64.asm crypto/rc4/rc4-md5-x86_64.asm crypto/rc4/rc4-x86_64.asm crypto/sha/keccak1600-x86_64.asm crypto/sha/sha1-mb-x86_64.asm crypto/sha/sha1-x86_64.asm crypto/sha/sha256-mb-x86_64.asm crypto/sha/sha256-x86_64.asm crypto/sha/sha512-x86_64.asm crypto/uplink-x86_64.asm crypto/whrlpool/wp-x86_64.asm crypto/x86_64cpuid.asm engines/e_padlock-x86_64.asm libcrypto.def libcrypto.rc libssl.def libssl.rc test/buildtest_aes.c test/buildtest_asn1.c test/buildtest_asn1t.c test/buildtest_async.c test/buildtest_bio.c test/buildtest_blowfish.c test/buildtest_bn.c test/buildtest_buffer.c test/buildtest_camellia.c test/buildtest_cast.c test/buildtest_cmac.c test/buildtest_cms.c test/buildtest_comp.c test/buildtest_conf.c test/buildtest_conf_api.c test/buildtest_crypto.c test/buildtest_ct.c test/buildtest_des.c test/buildtest_dh.c test/buildtest_dsa.c test/buildtest_dtls1.c test/buildtest_e_os2.c test/buildtest_ebcdic.c test/buildtest_ec.c test/buildtest_ecdh.c test/buildtest_ecdsa.c test/buildtest_engine.c test/buildtest_evp.c test/buildtest_hmac.c test/buildtest_idea.c test/buildtest_kdf.c test/buildtest_lhash.c test/buildtest_md4.c test/buildtest_md5.c test/buildtest_mdc2.c test/buildtest_modes.c test/buildtest_obj_mac.c test/buildtest_objects.c test/buildtest_ocsp.c test/buildtest_opensslv.c test/buildtest_ossl_typ.c test/buildtest_pem.c test/buildtest_pem2.c test/buildtest_pkcs12.c test/buildtest_pkcs7.c test/buildtest_rand.c test/buildtest_rand_drbg.c test/buildtest_rc2.c test/buildtest_rc4.c test/buildtest_ripemd.c test/buildtest_rsa.c test/buildtest_safestack.c test/buildtest_seed.c test/buildtest_sha.c test/buildtest_srp.c test/buildtest_srtp.c test/buildtest_ssl.c test/buildtest_ssl2.c test/buildtest_stack.c test/buildtest_store.c test/buildtest_symhacks.c test/buildtest_tls1.c test/buildtest_ts.c test/buildtest_txt_db.c test/buildtest_ui.c test/buildtest_whrlpool.c test/buildtest_x509.c test/buildtest_x509_vfy.c test/buildtest_x509v3.c tools/c_rehash.pl providers/common/der/der_digests_gen.c providers/common/der/der_dsa_gen.c providers/common/der/der_ec_gen.c providers/common/der/der_ecx_gen.c providers/common/der/der_rsa_gen.c providers/common/der/der_sm2_gen.c providers/common/der/der_wrap_gen.c

 PERLASM_SCHEME= auto
@@ -30,6 +30,9 @@ include/openssl/bio.h: include/openssl/bio.h.in configdata.pm
 include/openssl/cmp.h: include/openssl/cmp.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cmp.h.in" > $@
+include/openssl/comp.h: include/openssl/comp.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/comp.h.in" > $@
 include/openssl/cms.h: include/openssl/cms.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/cms.h.in" > $@
@@ -42,6 +45,9 @@ include/openssl/conf.h: include/openssl/conf.h.in configdata.pm
 include/openssl/configuration.h: include/openssl/configuration.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/configuration.h.in" > $@
+include/openssl/core_names.h: include/openssl/core_names.h.in configdata.pm
+       "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
+           "-omakefile" "include/openssl/core_names.h.in" > $@
 include/openssl/crmf.h: include/openssl/crmf.h.in configdata.pm
        "$(PERL)" "-I$(BLDDIR)" -Mconfigdata "util/dofile.pl" \
            "-omakefile" "include/openssl/crmf.h.in" > $@
@@ -93,6 +99,9 @@ include/openssl/ui.h: include/openssl/ui.h.in configdata.pm
 include/openssl/x509.h: include/openssl/x509.h.in configdata.pm
        $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
            "-omakefile" "include/openssl/x509.h.in" > $@
+include/openssl/x509_acert.h: include/openssl/x509_acert.h.in configdata.pm
+       $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
+           "-omakefile" "include/openssl/x509_acert.h.in" > $@
 include/openssl/x509_vfy.h: include/openssl/x509_vfy.h.in configdata.pm
        $(PERL) -I$(BLDDIR) -Mconfigdata util/dofile.pl \
            "-omakefile" "include/openssl/x509_vfy.h.in" > $@

I haven't figured out where to put/generate include/internal/param_names.h.in. Or the providers/common/* files.

Unfortunately regenerating the config with the above diff, still fails to build:

In file included from ../deps/openssl/openssl/providers/common/digest_to_nid.c:16:
../deps/openssl/openssl/providers/common/include/prov/securitycheck.h:13:10: fatal error: fips/fipsindicator.h: No such file or directory
   13 | #include "fips/fipsindicator.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [deps/openssl/openssl.target.mk:1327: /home/rlau/sandbox/github/node/out/Release/obj.target/openssl/deps/openssl/openssl/providers/common/digest_to_nid.o] Error 1
make[1]: *** Waiting for unfinished jobs....
In file included from ../deps/openssl/openssl/providers/common/securitycheck.c:21:
../deps/openssl/openssl/providers/common/include/prov/securitycheck.h:13:10: fatal error: fips/fipsindicator.h: No such file or directory
   13 | #include "fips/fipsindicator.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from ../deps/openssl/openssl/providers/common/securitycheck_default.c:16:
../deps/openssl/openssl/providers/common/include/prov/securitycheck.h:13:10: fatal error: fips/fipsindicator.h: No such file or directory
   13 | #include "fips/fipsindicator.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [deps/openssl/openssl.target.mk:1327: /home/rlau/sandbox/github/node/out/Release/obj.target/openssl/deps/openssl/openssl/providers/common/securitycheck.o] Error 1
make[1]: *** [deps/openssl/openssl.target.mk:1327: /home/rlau/sandbox/github/node/out/Release/obj.target/openssl/deps/openssl/openssl/providers/common/securitycheck_default.o] Error 1

fips/fipsindicator.h is not a generated file, so that probably indicates we need to update an include path somewhere and/or copy that file somewhere under our config.

richardlau avatar Apr 11 '25 01:04 richardlau

@jasnell

Ummmm..... good lord. It's going to be quite difficult to review this, unfortunately. The GitHub UI hangs and crashes whenever I try to open the changes tab.

Hopefully https://github.com/nodejs/node/pull/57835 will already improve things a bit.

targos avatar Apr 11 '25 15:04 targos

Simply because I'm not able to go actually look at the files changed list yet without my browser crashing, I just wanted to ask... does this PR make any changes to the ncrypto or src/crypto files or is it JUST updating the dependency so far?

jasnell avatar Apr 12 '25 17:04 jasnell

Apart from https://github.com/nodejs/node/pull/57795/commits/2d17835293f2ace9e2cd47d094b095a272b31ba6, the other commits only touch deps/openssl and are entirely auto-generated by the update script.

targos avatar Apr 12 '25 18:04 targos

New problems:

../../deps/openssl/openssl/providers/implementations/signature/ml_dsa_sig.c:21:10: fatal error: 'prov/der_ml_dsa.h' file not found
   21 | #include "prov/der_ml_dsa.h"
      |          ^~~~~~~~~~~~~~~~~~~
...
../../deps/openssl/openssl/providers/implementations/signature/slh_dsa_sig.c:17:10: fatal error: 'prov/der_slh_dsa.h' file not found
   17 | #include "prov/der_slh_dsa.h"
      |          ^~~~~~~~~~~~~~~~~~~~
1 error generated.

These headers must be generated from .h.in files, but they are not in the openssl/include directory. Someone will probably have to work on the generate_headers.pl script.

targos avatar Apr 15 '25 09:04 targos

These headers must be generated from .h.in files, but they are not in the openssl/include directory. Someone will probably have to work on the generate_headers.pl script.

https://github.com/nodejs/node/commit/5222c919b4cc4575e98db7f98097ccb53bdc0749 (also have to update the Windows Makefiles).

Now fails

../deps/openssl/openssl/providers/implementations/ciphers/ciphercommon_gcm.c:18:10: fatal error: internal/param_names.h: No such file or directory
   18 | #include "internal/param_names.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

which isn't wholly unexpected

I haven't figured out where to put/generate include/internal/param_names.h.in.

I haven't pushed https://github.com/nodejs/node/commit/5222c919b4cc4575e98db7f98097ccb53bdc0749 to this PR as I've run into a problem when I tried to rebase my working directory to this PR -- (re)generating the arch specific files fails for me:

make: Entering directory '/node/deps/openssl/config'
cd ../openssl; CONFIGURE_CHECKER_WARN=1 CC=gcc perl ./Configure no-comp no-shared no-afalgeng enable-ssl-trace enable-fips aix64-gcc-as;
Configuring OpenSSL version 3.5.0 for target aix64-gcc-as
Using os-specific seed configuration

Failure!  Makefile wasn't produced.
Please read INSTALL.md and associated NOTES-* files.  You may also have to
look over your available compiler tool chain or change your configuration.

Something went wrong with /node/deps/openssl/openssl/doc/build.info: No such file or directory
make: *** [Makefile:54: aix64-gcc-as] Error 2
make: Leaving directory '/node/deps/openssl/config'

which I'm assuming is failing because https://github.com/nodejs/node/pull/57835 removed the doc directory.

richardlau avatar Apr 25 '25 18:04 richardlau

I've opened https://github.com/nodejs/node/pull/58099 and https://github.com/nodejs/node/pull/58100. Hopefully these will be easier to review (neither of them include the actual OpenSSL 3.5 update which changes a lot of files).

richardlau avatar May 01 '25 17:05 richardlau

It's weird. https://github.com/nodejs/node/pull/57835 is included in the last force-push of this PR, which was generated by the update workflow on my fork.

targos avatar May 01 '25 18:05 targos

https://github.com/targos/node/actions/runs/14438290543/job/40483070056

targos avatar May 01 '25 18:05 targos

It's weird. #57835 is included in the last force-push of this PR, which was generated by the update workflow on my fork.

Well for me, on current main (c240c03013b2b5da76d7820755c8fc0f95f0ba12) (without any OpenSSL 3.5 related changes) just attempting to regenerate the OpenSSL config fails in the same way as https://github.com/nodejs/node/pull/57795#issuecomment-2831109945:

make: Entering directory '/node/deps/openssl/config'
cd ../openssl; CONFIGURE_CHECKER_WARN=1 CC=gcc perl ./Configure no-comp no-shared no-afalgeng enable-ssl-trace enable-fips aix64-gcc-as;
Configuring OpenSSL version 3.0.16 for target aix64-gcc-as
Using os-specific seed configuration

Failure!  Makefile wasn't produced.
Please read INSTALL.md and associated NOTES-* files.  You may also have to
look over your available compiler tool chain or change your configuration.

Something went wrong with /node/deps/openssl/openssl/doc/build.info: No such file or directory
make: *** [Makefile:54: aix64-gcc-as] Error 2
make: Leaving directory '/node/deps/openssl/config'

richardlau avatar May 01 '25 18:05 richardlau

Ah, that's for OpenSSL 3.0. I didn't think about testing my change with it, sorry! Can we maybe do the update to 3.5 without the revert and only prevent the deletions from landing on earlier release lines?

targos avatar May 01 '25 19:05 targos

Ah, that's for OpenSSL 3.0. I didn't think about testing my change with it, sorry! Can we maybe do the update to 3.5 without the revert and only prevent the deletions from landing on earlier release lines?

It happens for me with both 3.0 and 3.5 🤷. Please feel free to pick the commits from https://github.com/nodejs/node/pull/58100 (we'll need the test fix and the other changes in the second commit which is commits from this PR plus changes for the remaining files). But I won't be able to easily help out in the future if I'm unable to rerun the config generation.

richardlau avatar May 01 '25 19:05 richardlau

FWIW I'm running the Dockerfile to regenerate the config, as per https://github.com/nodejs/node/blob/c240c03013b2b5da76d7820755c8fc0f95f0ba12/Makefile#L1654-L1656 but it should be equivalent to https://github.com/nodejs/node/blob/c240c03013b2b5da76d7820755c8fc0f95f0ba12/tools/dep_updaters/update-openssl.sh#L79

richardlau avatar May 01 '25 19:05 richardlau

FWIW I'm going to be out until Tuesday, but I don't think there's a rush to get OpenSSL 3.5 landed as it shouldn't be semver-major. I don't actually care too much how we land the OpenSSL 3.5 update in Node.js, as long as it happens. I would suggest that https://github.com/nodejs/node/pull/58100 should be easier to review as it doesn't contain the OpenSSL update itself (so doesn't break the GitHub web UI).

Since I separated the revert (https://github.com/nodejs/node/pull/58099) from the test update and config generation changes (https://github.com/nodejs/node/pull/58100), one possible option is to land https://github.com/nodejs/node/pull/58100 and then run the updater GitHub workflow. If that works then great, we don't need the revert and can close it without landing. If it doesn't work we can then land the revert and then try the updater workflow again.

richardlau avatar May 02 '25 00:05 richardlau

We might have another wrinkle to solve. It looks like running the config generation on Ubuntu 24.04 (e.g. using the GitHub workflow) produces different assembly files than when run through deps/openssl/config/Dockerfile (Ubuntu 20.04) due to the different versions of GNU assembler: https://github.com/nodejs/node/pull/58097#issuecomment-2846127979

Unfortunately using Ubuntu 24.04 (and GNU assembler >=2.40) appears to cause OpenSSL's generators to generate assembly code containing instructions that cannot be compiled by the older versions of GNU assembler in the various platforms we run in Jenkins.

Possible ways around it might be to

  • Pin the workflow to older Ubuntu (e.g. 22.04). But GH will eventually sunset that runner.
  • Run the config generation with the Dockerfile (we could also update it from Ubuntu 20.04 to Ubuntu 22.04).

richardlau avatar May 02 '25 02:05 richardlau