Undefined symbols for architecture arm64 on swift 6.1

[ZirgVoice]

On Swift 6.1, when building, I get an error:

error: link command failed with exit code 1 (use -v to see invocation)
Undefined symbols for architecture arm64:
  "_RAND_priv_bytes", referenced from:
      _rd_kafka_UserScramCredentialUpsertion_new in rdkafka_admin.c.o
  "_SSL_CTX_set_cert_cb", referenced from:
      _rd_kafka_ssl_set_certs in rdkafka_ssl.c.o
  "_SSL_CTX_use_cert_and_key", referenced from:
      _rd_kafka_ssl_set_certs in rdkafka_ssl.c.o
  "_SSL_certs_clear", referenced from:
      _rd_kafka_ssl_cert_callback in rdkafka_ssl.c.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

MacOS: 15.4 Xcode: 16.3

[vsarunas]

Since https://github.com/swift-server/swift-kafka-client/pull/187, SSL dependency is no longer provided internally and an external system library needs to be installed:

brew install libressl

[ZirgVoice]

@vsarunas it didn't help, the error remained the same

[vsarunas]

I have these:

brew list | grep ssl
libressl
[email protected]
openssl@3

And also pkgconf.

[ZirgVoice]

I have the same list in brew list | grep ssl I installed pkgconf but it didn't help either

[mimischi]

Hey all, thanks for reporting!

I'm unable to reproduce the issue you're seeing on my arm64 machines. I just set up a new machine with macOS 15.4 (24E248), cloned this repository and installed libressl through brew (no pkgconf). I can build swift-kafka-client with both the swift CLI and within Xcode (16.3 / 16E140) using Swift 6.1. Building also works on another machine that only has openssl@3 and pkgconf.

Could you share some minimal project for me to try and reproduce your issue?

[ZirgVoice]

I created a new vapor project, via vapor new test and just added swift-kafka-client to Package, when trying to build I get an error. The problem does not seem to be related to swift 6.1, since there is no error when switching to version 1.0.0-alpha.7

[mimischi]

Building in a Vapor project works for me as well. Are you trying to build from within Xcode or using the swift CLI? If the latter, could you build again while setting all the corresponding environment variables?

PATH="$(brew --prefix libressl)/bin:$PATH" LDFLAGS="-L$(brew --prefix libressl)/lib" CPPFLAGS="-I$(brew --prefix ope libressl ssl)/include" swift build

[ZirgVoice]

The command you gave didn't work for me. This one worked: PATH="$(brew --prefix libressl)/bin:$PATH" LDFLAGS="-L$(brew --prefix libressl)/lib" CPPFLAGS="-I$(brew --prefix opa libressl sslh)/include" swift build. Your command gave me the following errors: No available formula with the name "ope". Did you mean opa or ode? and No available formula with the name "ssl". Did you mean sslh, isl or gsl?. But even with the command that worked, I got an error.

The build also doesn't work in xcode. The error in xcode is a little different, here it is:

Undefined symbols for architecture arm64:
  "_RAND_priv_bytes", referenced from:
      _rd_kafka_UserScramCredentialUpsertion_new in Crdkafka.o
  "_SSL_CTX_set_cert_cb", referenced from:
      _rd_kafka_ssl_set_certs in Crdkafka.o
  "_SSL_CTX_use_cert_and_key", referenced from:
      _rd_kafka_ssl_set_certs in Crdkafka.o
  "_SSL_certs_clear", referenced from:
      _rd_kafka_ssl_cert_callback in Crdkafka.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

UPDATE: I ​​tried completely reinstalling Homebrew. The result was the same.

[mimischi]

Oh, sorry. I somehow typo'd an ope into my command. CPPFLAGS wasn't set to the right path. Can you try again with the following?

PATH="$(brew --prefix libressl)/bin:$PATH" LDFLAGS="-L$(brew --prefix libressl)/lib" CPPFLAGS="-I$(brew --prefix libressl ssl)/include" swift build

And if this isn't working, would you mind sharing the outputs of:

brew list | grep ssl

as well as

brew list | grep ssl | xargs -I{} brew info {}

Interested to verify what SSL packages you've got installed.

[ZirgVoice]

Error Error: No available formula with the name "ssl". Did you mean sslh, isl or gsl? stayed.

Result brew list | grep ssl:

libressl
openssl@3

Result brew list | grep ssl | xargs -I{} brew info {}:

==> libressl: stable 4.0.0 (bottled), HEAD [keg-only]
Version of the SSL/TLS protocol forked from OpenSSL
https://www.libressl.org/
Installed
/opt/homebrew/Cellar/libressl/4.0.0 (3,479 files, 13.8MB)
  Poured from bottle using the formulae.brew.sh API on 2025-04-03 at 23:12:03
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/lib/libressl.rb
License: OpenSSL
==> Dependencies
Required: ca-certificates ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /opt/homebrew/etc/libressl/certs

and run
  /opt/homebrew/opt/libressl/bin/openssl certhash /opt/homebrew/etc/libressl/certs

libressl is keg-only, which means it was not symlinked into /opt/homebrew,
because it conflicts with OpenSSL.

If you need to have libressl first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/libressl/bin:$PATH"' >> ~/.zshrc

For compilers to find libressl you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/libressl/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/libressl/include"

For pkg-config to find libressl you may need to set:
  export PKG_CONFIG_PATH="/opt/homebrew/opt/libressl/lib/pkgconfig"
==> Analytics
install: 395 (30 days), 1,281 (90 days), 7,726 (365 days)
install-on-request: 370 (30 days), 1,200 (90 days), 7,124 (365 days)
build-error: 1 (30 days)
==> openssl@3: stable 3.4.1 (bottled)
Cryptography and SSL/TLS Toolkit
https://openssl-library.org
Installed
/opt/homebrew/Cellar/openssl@3/3.4.1 (7,236 files, 33.4MB) *
  Poured from bottle using the formulae.brew.sh API on 2025-04-03 at 23:13:32
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/o/[email protected]
License: Apache-2.0
==> Dependencies
Required: ca-certificates ✔
==> Caveats
A CA file has been bootstrapped using certificates from the system
keychain. To add additional certificates, place .pem files in
  /opt/homebrew/etc/openssl@3/certs

and run
  /opt/homebrew/opt/openssl@3/bin/c_rehash
==> Analytics
install: 393,025 (30 days), 1,190,342 (90 days), 5,029,665 (365 days)
install-on-request: 57,061 (30 days), 174,009 (90 days), 674,772 (365 days)
build-error: 9,505 (30 days)

[mimischi]

And.. I typo'd my command again. Thanks for the output @ZirgVoice. Let's start fresh and re-use the output from brew to export the corresponding flags:

export LDFLAGS="-L/opt/homebrew/opt/libressl/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libressl/include"
swift build

[vsarunas]

That is strange, I do not export these.

I only have brew in PATH.

[mimischi]

I've managed to reproduce the issue. In an environment where all three, libressl, openssl@3 and pkgconf are installed, I'm unable to build a Vapor application that depends on swift-kafka-client. Seeing the same compilation error as @ZirgVoice. swift-kafka-client is building fine by itself (checked out main from this repository).

@vsarunas I've installed [email protected] (after jumping through hoops, as it's seemingly disabled in brew for anyone trying to install it nowadays), and I'm also seeing the same issues. I now wonder what's special about your system. When you say that it works for you, is that just building swift-kafka-client by itself or within some application that does not depend on vapor?

[ZirgVoice]

I removed openssl@3 and pkgconf but still got the error. The error disappeared only after removing libressl. It turns out that there is no error only when openssl@3 and pkgconf are installed and libressl is not installed. This works in swift CLI, but in Xcode I get the error 'openssl/ssl.h' file not found

UPDATE: This helped with the error in Xcode

cd /Library/Developer/CommandLineTools/usr/include
sudo ln -s /opt/homebrew/opt/openssl@3/include/openssl

[mimischi]

@ZirgVoice can you try again using latest main?

[ZirgVoice]

@mimischi didn't help. Now, with any combination, I get the error 'openssl/ssl.h' file not found both in the console and in xcode. I had to go back to version 1.0.0-alpha.7

[mimischi]

@ZirgVoice What's the output of brew list | grep ssl on your machine when the build is failing?

[ZirgVoice]

@mimischi

❯ brew list | grep ssl
libressl
openssl@3

but the error is present even after removing libressl

[ZirgVoice]

I'm completely confused, there is no error in the terminal, but there is in xcode

[t089]

I'm completely confused, there is no error in the terminal, but there is in xcode

Try quitting Xcode and the launching it again from the Terminal:

$ xed .

[ZirgVoice]

it got even weirder. I restarted xcode and cleaned the build folder. Now the dev build works, but the release gives a bunch of errors