Add OpenSSL 3 support

[FedericoCeratto]

Drop support for versions below 1.1.0 Move away from deprecated functions Fixes: #19604

[Yardanico]

So are you also planning on removing LibreSSL support as it relies on OpenSSL 1.0 APIs? It's really nice that you're doing the work with bringing up OpenSSL 3, but I think dropping support for OpenSSL < 1.1.0 is not a good idea.

[FedericoCeratto]

@Yardanico supporting OpenSSL 3, 1.1, 1.0, 0.9 and LibreSSL, both with dynamic and static linking (and across multiple OSes) creates a combinatorial explosion of test scenarios, making it really difficult to ensure the libraries are used securely. IMHO we should try to reduce the complexity in the wrapper. Maybe even remove a good fraction of unused procs. See #19604 for more comments.

[heinthanth]

No Luck ... the same error.

I think it's due to macOS SIP. It don't pass VARIABLES starts with DYLD_ to subprocess.

DYLD_LIBRARY_PATH variables exists. But when I execute levelOne.sh that var is wiped out. So, u can't set DYLD_* variable outside and use inside script that's the main issue. Other than that, the pull request works fine!

https://stackoverflow.com/questions/35568122/why-isnt-dyld-library-path-being-propagated-here https://github.com/rbenv/rbenv/issues/962

I found those issues.

[heinthanth]

Here are the proofs

I've a script to run compiled nimble at bin dir. When I run that nimble, it works ( since var is passed to nimble process ) But when I run it via runNimble.sh, didn't work ( since var is wiped out ). PS: I compiled nimble with -d:nimDebugDlOpen to debug.

[FedericoCeratto]

Env vars get lost might get lost in different locations in nimInternalBuildKochAndRunCI. Even nim r foo.nim drops them. Besides, having to add special env vars for OSX would be an annoying requirement for end users. SSL should work out of the box.

[FedericoCeratto]

The library import on Mac OS X is now fixed.

[FedericoCeratto]

@ringabout @Araq the remaining issue seems to be a timeout due to slow CI. Do you think the PR can be reviewed and merged?

[ringabout]

the remaining issue seems to be a timeout due to slow CI.

I restarted the failed CI.

[Araq]

One more attempt before I'll merge it anyway.

[heinthanth]

Failed CI is due to timeout, I guess. Anyway, nice work @FedericoCeratto. Congrats!

[FedericoCeratto]

@Araq is there any plan to make a minor release to add support for OpenSSL 3? A number of users are being impacted by this.

[Araq]

@Araq is there any plan to make a minor release to add support for OpenSSL 3?

Yes, we'll make a release for this.

[dom96]

Small suggestion: since we don't seem to have full automated tests for this, can you describe how you've tested this PR locally manually? I'm sure it will help us for the future, even if it is you doing the testing again :)

[FedericoCeratto]

@dom96 I've tested openssl 3 locally and in CI runs using ubuntu-22.04 on my fork https://github.com/FedericoCeratto/Nim/pull/4 The image contains only openssl 3. I didn't add a new set of CI runs here because it's already quite heavy. Maybe we could split some of the package CI runs across ubuntu-22.04 and ubuntu-20.04 as a compromise. If you are referring to CLI commands to run tests, there are examples in the openssl.nim file.

(hah, today some test certs expired: https://github.com/chromium/badssl.com/issues/510#issue-1332362847 :laughing: )

[ringabout]

(hah, today some test certs expired: https://github.com/chromium/badssl.com/issues/510#issue-1332362847 😆 )

Yeah, I had to workaround it here: https://github.com/nim-lang/Nim/pull/20181

[FedericoCeratto]

@Araq finally the CI is all green :)

[github-actions[bot]]

Thanks for your hard work on this PR! The lines below are statistics of the Nim compiler built from 2dcfd732609a2cfa805e5a94cc105399a2f18632

Hint: mm: orc; threads: on; opt: speed; options: -d:release 163641 lines; 12.692s; 841.465MiB peakmem

[arnetheduck]

These changes break CI on macos 11 for many of our projects that previously worked without having to install additional dependencies:

Nim Compiler Version 1.7.1 [MacOSX: amd64]
Compiled at 2022-08-26
Copyright (c) 2006-2022 by Andreas Rumpf

git hash: ea44c5cfed21951feb5978b74fbc6cdb24f54ac2
active boot switches: -d:release
could not load: libcrypto.1.1.dylib
(compile with -d:nimDebugDlOpen for more information)

https://github.com/status-im/nim-web3/runs/8027875139?check_suite_focus=true

[hkl615]

Hiiii

[hkl615]

FedericoCeratto:openssl3