php-rdkafka icon indicating copy to clipboard operation
php-rdkafka copied to clipboard

Problem installing PHP 8.3 extension on Windows

Open andrey-helldar opened this issue 1 year ago • 24 comments

Description

I'm trying to connect the rdkafka 6.0.3 extension to PHP 8.3.9, but I'm getting an error:

$ php -v
PHP Warning:  PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0
PHP 8.3.9 (cli) (built: Jul  2 2024 18:17:57) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.3.9, Copyright (c) Zend Technologies
    with Zend OPcache v8.3.9, Copyright (c), by Zend Technologies
    with Xdebug v3.3.1, Copyright (c) 2002-2023, by Derick Rethans

$ which php
/d/modules/php/php

The rdkafka files are located in the following paths:

/d/modules/php/ext/php_rdkafka.dll
/d/modules/php/librdkafka.dll
/d/modules/php/librdkafka++.dll
  • My php.ini: https://pastebin.com/nbqTfSuP
  • https://phpext.phptools.online/extension/networking/rdkafka-180

That said, there is no problem with the extension installed in Ubuntu WSL.

I also tried deleting the librdkafka++.dll file (it was inside the archive), but that didn't help.

What could be the problem?

For PHP 8.1 and 8.2, the same problem.

php-rdkafka Version

php-rdkafka 6.0.3

librdkafka Version

No response

PHP Version

PHP 8.3.9

Operating System

Windows 11

Kafka Version

No response

andrey-helldar avatar Jul 08 '24 14:07 andrey-helldar

You should not really use .dll files provided by sources that are hardly verified. I do not know the owner of phpext.phptools.online, but I would be hard pressed to consider it remotely safe.

Try https://pecl.php.net/package/rdkafka/6.0.3/windows instead.

php.net is under PHP group, so unless something goes very wrong, you should use those. A checksum check should also be something you'd do, just to be sure.

It is executable code you're allowing to run, so be careful :)

Steveb-p avatar Jul 08 '24 15:07 Steveb-p

Didn't work with the pecl option either :(

Helldar@HellPC MINGW64 /d/domains
$ php -v
PHP Warning:  PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0
PHP 8.3.9 (cli) (built: Jul  2 2024 18:17:57) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.3.9, Copyright (c) Zend Technologies
    with Zend OPcache v8.3.9, Copyright (c), by Zend Technologies
    with Xdebug v3.3.1, Copyright (c) 2002-2023, by Derick Rethans

In the screenshot on the left is the phptools variant and on the right is pecl: image

I've also tried specifying different links to the extension:

extension = rdkafka
;extension = rdkafka.dll
;extension = php_rdkafka
;extension = php_rdkafka.dll

andrey-helldar avatar Jul 08 '24 20:07 andrey-helldar

Detailed steps to reproduce the problem:

  1. Download latest Non-Thread Safe PHP 8.3: https://windows.php.net/downloads/releases/php-8.3.9-nts-Win32-vs16-x64.zip
  2. Unzipped the archive into the "d:\php" folder.
  3. Renamed php.ini-development with php.ini
  4. Download rdkafka extension from PECL: https://downloads.php.net/~windows/pecl/releases/rdkafka/6.0.3/php_rdkafka-6.0.3-8.3-nts-vs16-x64.zip
  5. Move rdkafka's files to PHP directory:
    d:\php\ext\php_rdkafka.dll
    d:\php\librdkafka.dll
    d:\php\librdkafka++.dll
    d:\php\php_rdkafka.pdb
    
  6. Uncommented the extension location parameter in php.ini:
    ; On windows:
    extension_dir = "ext"
    
  7. Added the rdkafka extension:
    extension = rdkafka
    
  8. Open the terminal
  9. Executed the following console commands:
    cd d:\php
    ./php --version
    

Result:

Helldar@HellPC MINGW64 /d/php
$ ./php --version
PHP Warning:  PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0
PHP 8.3.9 (cli) (built: Jul  2 2024 18:17:57) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.3.9, Copyright (c) Zend Technologies

andrey-helldar avatar Jul 09 '24 09:07 andrey-helldar

Trying the same steps in the Windows Sandbox.

  1. I get the error vcruntime140.dll file is unavailable
  2. I install Visual C++ Redistributable 2019 from https://aka.ms/vs/17/release/vc_redist.x64.exe
c:\php>php.exe --version
PHP Warning:  PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0
PHP 8.3.9 (cli) (built: Jul  2 2024 18:17:57) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.3.9, Copyright (c) Zend Technologies

andrey-helldar avatar Jul 09 '24 09:07 andrey-helldar

Checking on other versions:

PHP 8.1.29

c:\php>php.exe --version
PHP 8.1.29 (cli) (built: Jun  5 2024 10:43:14) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.1.29, Copyright (c) Zend Technologies

c:\php>php.exe -m
[PHP Modules]
...
rdkafka
...

PHP 8.2.21

c:\php>php.exe --version
PHP Warning:  PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0
PHP 8.2.21 (cli) (built: Jul  2 2024 14:00:59) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.2.21, Copyright (c) Zend Technologies

PHP 8.3.9

c:\php>php.exe --version
PHP Warning:  PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'rdkafka' (tried: ext\rdkafka (The specified module could not be found), ext\php_rdkafka.dll (The specified module could not be found)) in Unknown on line 0
PHP 8.3.9 (cli) (built: Jul  2 2024 18:17:57) (NTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.3.9, Copyright (c) Zend Technologies

andrey-helldar avatar Jul 09 '24 09:07 andrey-helldar

I have the same problem!

wgole avatar Jul 11 '24 03:07 wgole

Hello, I am facing the same issue, did anybody found a solution so far ?

Adham-A avatar Jul 17 '24 07:07 Adham-A

Hello, I am facing the same issue, did anybody found a solution so far ?

The problem is solved by compiling the library to a new version of PHP. I haven't found someone who can do that yet

andrey-helldar avatar Jul 17 '24 10:07 andrey-helldar

Hello, I am facing the same issue, did anybody found a solution so far ?

The problem is solved by compiling the library to a new version of PHP. I haven't found someone who can do that yet

Why is that ? There are no guidelines on how to do it ? Or is it a more deep technical issue ? I might be able to help

Adham-A avatar Jul 17 '24 11:07 Adham-A

Hello, I am facing the same issue, did anybody found a solution so far ?

The problem is solved by compiling the library to a new version of PHP. I haven't found someone who can do that yet

Why is that ? There are no guidelines on how to do it ? Or is it a more deep technical issue ? I might be able to help

When compiling a DLL, you need to consider the environment and its settings for which the compilation is being done.

Given the fact that the DLL runs on PHP 8.1, I can assume that either the PHP 8.2 and 8.3 versions were compiled with PHP 8.1 settings, or with incorrect settings, or the compiler did not take into account any nuances when building for PHP 8.2 and 8.3.

For example, the same tests (https://github.com/arnaud-lb/php-rdkafka/actions/runs/9445153156) - they were run on different environments. So there is a nuance somewhere.

Either way, you can find out by running the compiler in a customized development environment.

Translated with DeepL.com (free version)

andrey-helldar avatar Jul 17 '24 12:07 andrey-helldar

I successfully added the library to my project to work on one of my Windows environments using WampServer and PHP 8.2. However, when I attempted to set it up on my second Windows environment, with the same versions, it did not work.

I used the Dependency Walker tool from https://github.com/lucasg/Dependencies to identify any missing dependencies in the DLL files. Despite adding the missing dependencies, the setup still fails to work on the second environment.

Does anyone know if there is a requirment list of Windows DLL to make it work ?

Adham-A avatar Jul 18 '24 10:07 Adham-A

I successfully added the library to my project to work on one of my Windows environments using WampServer and PHP 8.2. However, when I attempted to set it up on my second Windows environment, with the same versions, it did not work.

I used the Dependency Walker tool from https://github.com/lucasg/Dependencies to identify any missing dependencies in the DLL files. Despite adding the missing dependencies, the setup still fails to work on the second environment.

Does anyone know if there is a requirment list of Windows DLL to make it work ?

The latest version of this project was released on October 25, 2021. PHP 8.1 was released exactly one month later, on November 25.

I doubt that this project will correctly search for missing dependencies for PHP 8.1, 8.2 and 8.3.

andrey-helldar avatar Jul 18 '24 11:07 andrey-helldar

I've looked around Google there are no good looking alternatives in PHP, I will try to dig into the project then

Adham-A avatar Jul 18 '24 13:07 Adham-A

Also not working for me. Got the DLLs from PECL and they Just Do Not Work. Hope to see a fix here soon!

doomdudeblue avatar Aug 05 '24 19:08 doomdudeblue

Same problem

neokofg avatar Aug 15 '24 06:08 neokofg

same problem

dhanyn10singapay avatar Aug 16 '24 07:08 dhanyn10singapay

now i change to python in microservice to get kafka logs instead of using rdkafka. If anyone need for tutorial, you can ask me directly

dhanyn10singapay avatar Aug 21 '24 07:08 dhanyn10singapay

The problem is that the librdkafka library is built for a different version of PHP.

Is there anyone who knows how to compile it for the right version of PHP? The last stable release is 2.5.0 from July 10 (https://github.com/confluentinc/librdkafka/releases).

andrey-helldar avatar Aug 21 '24 08:08 andrey-helldar

Download php 8.1 for Windows, copy libssl-1_1.dll, libcrypto-1_1.dll from php 8.1 to php 8.2 and it will work.

phglong1999 avatar Oct 02 '24 12:10 phglong1999

@phglong1999, but it won't work for 8.3 anyway.

andrey-helldar avatar Oct 02 '24 13:10 andrey-helldar

Managed to solve this for PHP 8.3 on Windows 11 after some trial and error

  1. Download the Nuget package from https://www.nuget.org/packages/librdkafka.redist/ (click on "Download package")
  2. Open the downloaded file via 7zip or any ZIP file browser.
  3. Copy the following files (libcurl.dll, librdkafka.dll, msvcp140.dll, vcruntime140.dll, zlib1.dll, zstd.dll) from runtimes\win-x64\native\ folder into your PHP root folder same level as php.ini
  4. Run php -v / -m again, php-rdkafka should be added to your PHP modules successfully now

leexin avatar Oct 07 '24 02:10 leexin

@andrey-helldar I have installed phprdkafka on version 8.2

auto2810321 avatar Oct 19 '24 08:10 auto2810321

@andrey-helldar Please follow the instructions to install phprdkafka bth and install OpenSSL. Please install the correct version 1.1 and do not install a higher version. If you have any errors please let me know https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe

auto2810321 avatar Oct 19 '24 08:10 auto2810321

@andrey-helldar Please follow the instructions to install phprdkafka bth and install OpenSSL. Please install the correct version 1.1 and do not install a higher version. If you have any errors please let me know https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe

I see two problems in this case:

  1. I am not using PHP 8.2. The project uses version 8.3.
  2. Download an exe file from an obscure site? No, thanks. If I want to download openssl, I will only do it from the official site.

andrey-helldar avatar Oct 19 '24 10:10 andrey-helldar

Managed to solve this for PHP 8.3 on Windows 11 after some trial and error

1. Download the Nuget package from https://www.nuget.org/packages/librdkafka.redist/ (click on "Download package")

2. Open the downloaded file via 7zip or any ZIP file browser.

3. Copy the following files (libcurl.dll, librdkafka.dll, msvcp140.dll, vcruntime140.dll, zlib1.dll, zstd.dll) from runtimes\win-x64\native\ folder into your PHP root folder same level as php.ini

4. Run php -v / -m again, php-rdkafka should be added to your PHP modules successfully now

This didn't work for me at all, with PHP 8.3.16

Is there anything else to do, perhaps regarding php.ini file ?

titoshadow avatar Feb 03 '25 22:02 titoshadow

Managed to solve this for PHP 8.3 on Windows 11 after some trial and error

1. Download the Nuget package from https://www.nuget.org/packages/librdkafka.redist/ (click on "Download package")

2. Open the downloaded file via 7zip or any ZIP file browser.

3. Copy the following files (libcurl.dll, librdkafka.dll, msvcp140.dll, vcruntime140.dll, zlib1.dll, zstd.dll) from runtimes\win-x64\native\ folder into your PHP root folder same level as php.ini

4. Run php -v / -m again, php-rdkafka should be added to your PHP modules successfully now

This didn't work for me at all, with PHP 8.3.16

Is there anything else to do, perhaps regarding php.ini file ?

This goes without saying you need to import php_rdkafka.dll in your php.ini file and have php_rdkafka.dll inside your ext/ folder

in php.ini add: extension=php_rdkafka.dll

leexin avatar Feb 04 '25 00:02 leexin

Managed to solve this for PHP 8.3 on Windows 11 after some trial and error

  1. Download the Nuget package from https://www.nuget.org/packages/librdkafka.redist/ (click on "Download package")
  2. Open the downloaded file via 7zip or any ZIP file browser.
  3. Copy the following files (libcurl.dll, librdkafka.dll, msvcp140.dll, vcruntime140.dll, zlib1.dll, zstd.dll) from runtimes\win-x64\native\ folder into your PHP root folder same level as php.ini
  4. Run php -v / -m again, php-rdkafka should be added to your PHP modules successfully now

Works for PHP 8.4 as well

rogervila avatar Mar 14 '25 15:03 rogervila

Managed to solve this for PHP 8.3 on Windows 11 after some trial and error

  1. Download the Nuget package from https://www.nuget.org/packages/librdkafka.redist/ (click on "Download package")
  2. Open the downloaded file via 7zip or any ZIP file browser.
  3. Copy the following files (libcurl.dll, librdkafka.dll, msvcp140.dll, vcruntime140.dll, zlib1.dll, zstd.dll) from runtimes\win-x64\native\ folder into your PHP root folder same level as php.ini
  4. Run php -v / -m again, php-rdkafka should be added to your PHP modules successfully now

Works for PHP 8.4 as well

Works for PHP 8.4 and I only use librdkafka.dll (2351 KB). Thanks for your help.

ntquangkk avatar Apr 22 '25 03:04 ntquangkk

Installation php-rdkafka on Windows

(Updated June 2025 - Supports PHP 8.4 and below)

Prerequisites

  • PHP installed (version < 8.4 or 8.4)
  • Administrative access to modify PHP configuration
  • 7-Zip (or similar tool to extract .nupkg files)

Step 1: Download php_rdkafka.dll

  1. On the page PECL rdkafka page
  2. Click the DLL button matching your environment:
    • PHP version (e.g., 8.3, 8.2)
    • Architecture (x64 or x86)
    • Thread Safety (Recommended: Thread Safe (TS))
  3. Extract the downloaded .zip and locate php_rdkafka.dll.
  4. Move this file to your PHP extensions folder (typically php/ext).

Step 2: Install Dependencies (librdkafka)

  1. Download the librdkafka NuGet package:
    nuget.org/packages/librdkafka.redist (Click "Download package")
  2. Open the .nupkg file:
    • Use 7-Zip, WinRAR, or rename it the extension to .zip.
  3. Navigate to:
    runtimes\win-x64\native/  
    
  4. Copy these files to your PHP root directory (where php.ini is located):
    • librdkafka.dll (Required for all PHP versions)
    • For PHP < 8.4, also copy:
      • libcurl.dll
      • msvcp140.dll
      • vcruntime140.dll
      • zlib1.dll
      • zstd.dll

Note for PHP 8.4+ users: Only librdkafka.dll is required. The other dependencies are bundled with PHP.


Step 3: Enable the Extension

  1. Open php.ini (in your PHP root folder) and add:
    extension=php_rdkafka.dll
    
  2. Verify installation:
    php -v
    php -m | find "rdkafka"
    
    • If successful, rdkafka will appear in the module list.

Testing & Next Steps

  • Test the extension by running a Kafka script or using a library like:
    LaravelKafka (for Laravel projects)

Mathissou01 avatar Jun 18 '25 10:06 Mathissou01

@Mathissou01 It's works for PHP 8.3 and 8.4. Thank you!

andrey-helldar avatar Jun 18 '25 11:06 andrey-helldar