[4.x] Fix LDAP over SSL

[tatankat]

Pull Request for Issue (none created)

Summary of Changes

Convert negotiate TLS option to encryption protocol option to re-enable the use of ldap over ssl (ldaps). I am not too sure about the filename of the database changes, please review and comment.

Testing Instructions

Use an LDAP server with LDAPS. When entering the full ldap URI (ldaps://example.com) in the Host field in V3, it was working.

Actual result BEFORE applying this Pull Request

Joomla was trying to connect to ldap://ldaps://example.com When only entering the hostname, Joomla was trying to connect to ldap://example.com The prefix "ldap://" is added by the symfony library.

Expected result AFTER applying this Pull Request

Joomla connects to ldaps://example.com (only) when the SSL encryption protocol is selected - changed behavior wrt V3. Behavior for no encryption and TLS negotiation has not changed.

Documentation Changes Required

Possibly

[richard67]

I am not too sure about the filename of the database changes, please review and comment.

I can do that tomorrow or on weekend. The file names look ok at a first quick look, and SQL syntax and style looks ok, too, but I have to check the replace for the parameter value because it needs to be very careful with that. We have to make sure to really match the complete parameter. Am too tired today to do that now.

[tatankat]

@richard67 Can you please review? Is there anything else I can / have to do to make this PR and #37959 to be included in the first possible coming release?

[richard67]

@richard67 Can you please review? Is there anything else I can / have to do to make this PR and #37959 to be included in the first possible coming release?

Each PR needs 2 successful human tests. It could be hard to find testers for LDAP authentication since it's rarely used. So this alone might take some time.

For the other PR people can maybe use https://www.forumsys.com/2022/05/10/online-ldap-test-server/ , but I'm not sure if it is also suitable for this one for LDAPS.

[tatankat]

Thanks for the review and suggestions, I applied them. Now let's hope someone wants to test this...

[HLeithner]

This pull requests has automatically rebased to 4.2-dev.

[joomla-bot]

This pull requests has been automatically converted to the PSR-12 coding standard.

[HLeithner]

@tatankat 2 things

first can you change the sql files to 4.2.0 with date. 2nd do you have experience with ldap server and docker and tests^^ We would really need automated tests for ldap, so having a ldap server automatically provisioned with a users so joomla can login would be great but I don't know if something like this could be made easily.

[richard67]

@tatankat 2 things

first can you change the sql files to 4.2.0 with date.

@tatankat I suggest you rename both update SQL scripts to "4.2.0-2022-06-28.sql".

[tatankat]

@HLeithner

2nd do you have experience with ldap server and docker and tests^^ We would really need automated tests for ldap, so having a ldap server automatically provisioned with a users so joomla can login would be great but I don't know if something like this could be made easily.

I don't have any real-life experience with any possible combination of docker, php, unittests, ldap and joomla. But I do have at least some basic experience with any of them separately.

I researched a bit and to run a docker with openldap and non-encrypted, with ssl/tls or starttls communication (on the standard ports, but possible to change) and an account with a password to test, you can use (see also https://github.com/bitnami/bitnami-docker-openldap):

docker run --rm --name openldap --env LDAP_ADMIN_USERNAME=admin --env LDAP_ADMIN_PASSWORD=adminpassword --env LDAP_USERS=customuser --env LDAP_PASSWORDS=custompassword --publish 389:1389 --publish 636:1636 --env LDAP_ENABLE_TLS=yes --env LDAP_TLS_CERT_FILE=/opt/bitnami/certs/ldapcert.crt --env LDAP_TLS_KEY_FILE=/opt/bitnami/certs/ldapcert.key --env LDAP_TLS_CA_FILE=/opt/bitnami/certs/CA.crt -v <path_where_the_certs_and_key_are>:/opt/bitnami/certs --env BITNAMI_DEBUG=true -u $(id -u) bitnami/openldap:latest

If you can give me a basic structure in the Joomla testing structure to put the ldap tests, I am willing to give it a try (in a few weeks). I didn't test Joomla yet with this docker openldap.

[nsheehan]

We currently use J3 with LDAP (Apache DS) and are starting to do some testing with J4. We are willing to do some testing.

[tatankat]

Thanks for willing to test @nsheehan You're kindly invited to run some tests. Depending on what you're requirements are, you need this PR and #37959. To do ldap client side debugging, you need #38388 . All of them are merged in https://github.com/tatankat/joomla-cms/tree/patched. I am very keen to hear about your experiences.

Furthermore, I succeeded to use the docker openldap with Joomla. To successfully log in using the ldap user, it needs an email address set which you can add by: creating a file with contents:

dn: cn=customuser,ou=users,dc=example,dc=org
add: mail
mail: [email protected]
-

and executing:

ldapmodify -v -x -H ldap://localhost -f <the file's name> -D cn=admin,dc=example,dc=org -w adminpassword

So if anyone can tell me where to put the tests and which files I can use to get the structure from, I can create some automated tests.

[tatankat]

I've tried to create a package to ease testing of upgrading to V4 the LDAP patches, see: https://github.com/tatankat/joomla-cms/releases/tag/patched Do not use this on an instance if you plan to continue using it!

[richard67]

So if anyone can tell me where to put the tests and which files I can use to get the structure from, I can create some automated tests.

@tatankat When looking how unit tests have been done for the task plugins here https://github.com/joomla/joomla-cms/tree/4.2-dev/tests/Unit/Plugin/Task (we seem not to have any other unit tests for other plugin groups yet), I think you should create a folder "tests/Unit/Plugin/authentication" for the plugin group, and in that folder a folder "ldap", and in that one you put your test. The ldap plugin has not been changed to the new structure yet, so it will not need a sub-folder "Extension" like plugins in the "tests/Unit/Plugin/Task" folder have, and your folder will be named "ldap" and not "ldap", I think.

But maybe @laoneo has a better idea?

Or did you mean system tests?

[nickdring]

Hi, @tatankat I used your package to test the LDAP and unfortunately it's still not working. The only error messages I see every time I try to log in are of this kind, '2022-05-24T13:15:10+00:00 INFO 10.255.7.56 ldapfailure Username and password do not match, or you do not have an account yet.'

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/37962.}

[tatankat]

@HLeithner @richard67 I created some tests in https://github.com/tatankat/joomla-cms/commit/2ee0907bf49159c8fd9bfa905b8454e2fce4e31b As this adds another service (and docker image) in drone, I thinks it's best I wait approval before creating a PR. Can you approve? For now, the services will certainly fail, as the certificates are missing. Do you have an opinion on how to generate the certificates? Do you know where I can put them? The tests are usable as-is, but most certainly need improvements. @HLeithner thanks for the information. To answer your question, I think it is more an integration test, but I put it in the unit tests as you explained. @nickdring see my answer in #37959

[HLeithner]

@tatankat looks good. about the certificate, create a certificate that is valid for 50 years and add it to the tests/Codeception/_data/certs subdirectory. I'm not sure how the client can use the correct certificates...

Also remove 4.2 with DEPLOY_VERSION please

But please create a pull request and we can see how it goes.

thanks Harald

[VitaSati]

I have tested this item :white_check_mark: successfully on 9fd0f313fdfbc5a4516d85a4b402218ff0a703c6

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/37962.}

[Hackwar]

Can you please also change the tests to skip when LDAP is not available?

[tatankat]

@Hackwar #38775 may already solve part of your problem

[nickdring]

Hi, I updated to 4.2.3, and it still doesn't work for me, I don't know if the changes made here were incorporated into that release. @tatankat could you make me an installer for Joomla with all the patches you added, so I can try again? I'm following another thread on GitHub where other people are still having issues. Thanks!

[tatankat]

@nickdring the changes are not yet incorporated as there are not enough successful human tests... I have created a new update package with all fixes: https://github.com/tatankat/joomla-cms/releases/tag/4.2.3-patched

[Fedik]

@nickdring you also can download test build under each Pull Request

Follow the Details link https://ci.joomla.org/artifacts/joomla/joomla-cms/4.2-dev/37962/downloads/58382/ and download Development-Full_Package.zip or Development-Update_Package.zip for test.

[tatankat]

Important addition: if you only need the changes of 1 PR. The build I created contains the changes of all ldap related PRs.

[nickdring]

Great guys, I have installed on my local machine. I'll start trying and let you know soon.

[nickdring]

Hi, no luck I'm afraid. I switched logging on so here is the output of the variations I tried. #Fields: datetime priority clientip category message 2022-10-07T12:04:17+00:00 DEBUG ::1 ldap Creating LDAP session with options: {"host":"10.255.8.30","port":3269,"version":2,"referrals":true,"encryption":"ssl","debug":true} 2022-10-07T12:04:17+00:00 DEBUG ::1 ldap Creating LDAP session to connect to "ldaps://10.255.8.30:3269" while binding 2022-10-07T12:04:17+00:00 DEBUG ::1 ldap Direct binding to LDAP server with entered user dn "[email protected]" and user entered password 2022-10-07T12:04:17+00:00 ERROR ::1 ldap Can't contact LDAP server 2022-10-07T12:04:17+00:00 INFO ::1 ldapfailure Username and password do not match or you do not have an account yet. 2022-10-07T12:04:17+00:00 WARNING ::1 jerror Username and password do not match or you do not have an account yet. 2022-10-07T12:05:19+00:00 INFO ::1 controller Holding edit ID com_plugins.edit.plugin.90 Array ( [0] => 90 ) 2022-10-07T12:05:19+00:00 INFO ::1 controller Checking edit ID com_plugins.edit.plugin.90: 1 Array ( [0] => 90 ) 2022-10-07T12:05:23+00:00 INFO ::1 controller Holding edit ID com_plugins.edit.plugin.90 Array ( [0] => 90 ) 2022-10-07T12:05:23+00:00 INFO ::1 controller Checking edit ID com_plugins.edit.plugin.90: 1 Array ( [0] => 90 ) 2022-10-07T12:05:31+00:00 DEBUG ::1 ldap Creating LDAP session with options: {"host":"10.255.8.30","port":3269,"version":2,"referrals":true,"encryption":"tls","debug":true} 2022-10-07T12:05:31+00:00 DEBUG ::1 ldap Creating LDAP session to connect to "ldap://10.255.8.30:3269" while binding 2022-10-07T12:05:31+00:00 DEBUG ::1 ldap Direct binding to LDAP server with entered user dn "[email protected]" and user entered password 2022-10-07T12:05:31+00:00 ERROR ::1 ldap Could not initiate TLS connection: Success 2022-10-07T12:05:31+00:00 INFO ::1 ldapfailure Username and password do not match or you do not have an account yet. 2022-10-07T12:05:31+00:00 WARNING ::1 jerror Username and password do not match or you do not have an account yet. 2022-10-07T12:05:47+00:00 INFO ::1 controller Holding edit ID com_plugins.edit.plugin.90 Array ( [0] => 90 ) 2022-10-07T12:05:48+00:00 INFO ::1 controller Checking edit ID com_plugins.edit.plugin.90: 1 Array ( [0] => 90 ) 2022-10-07T12:05:55+00:00 DEBUG ::1 ldap Creating LDAP session with options: {"host":"10.255.8.30","port":3269,"version":3,"referrals":true,"encryption":"tls","debug":true} 2022-10-07T12:05:55+00:00 DEBUG ::1 ldap Creating LDAP session to connect to "ldap://10.255.8.30:3269" while binding 2022-10-07T12:05:55+00:00 DEBUG ::1 ldap Direct binding to LDAP server with entered user dn "[email protected]" and user entered password 2022-10-07T12:05:55+00:00 ERROR ::1 ldap Could not initiate TLS connection: Success 2022-10-07T12:05:55+00:00 INFO ::1 ldapfailure Username and password do not match or you do not have an account yet. 2022-10-07T12:05:55+00:00 WARNING ::1 jerror Username and password do not match or you do not have an account yet. 2022-10-07T12:06:12+00:00 INFO ::1 controller Holding edit ID com_plugins.edit.plugin.90 Array ( [0] => 90 ) 2022-10-07T12:06:13+00:00 INFO ::1 controller Checking edit ID com_plugins.edit.plugin.90: 1 Array ( [0] => 90 ) 2022-10-07T12:06:18+00:00 DEBUG ::1 ldap Creating LDAP session with options: {"host":"10.255.8.30","port":3269,"version":3,"referrals":true,"encryption":"ssl","debug":true} 2022-10-07T12:06:18+00:00 DEBUG ::1 ldap Creating LDAP session to connect to "ldaps://10.255.8.30:3269" while binding 2022-10-07T12:06:18+00:00 DEBUG ::1 ldap Direct binding to LDAP server with entered user dn "[email protected]" and user entered password 2022-10-07T12:06:18+00:00 ERROR ::1 ldap Can't contact LDAP server 2022-10-07T12:06:18+00:00 INFO ::1 ldapfailure Username and password do not match or you do not have an account yet. 2022-10-07T12:06:18+00:00 WARNING ::1 jerror Username and password do not match or you do not have an account yet.

[tatankat]

@nickdring you need to look at the ldap client debug messages (the one you provided only a part of in #37959, see https://github.com/joomla/joomla-cms/pull/37959#issuecomment-1246720311) You probably need TLS_REQCERT never (this should be clear from the ldap client debug log) and read the mark about it at https://github.com/joomla/joomla-cms/pull/37959#issuecomment-1247330045

[nickdring]

I'll put this patch build on a staging server so I'm sure its not a local machine problem with the ldap server

[noxidsoft]

Hi team, This is also an ongoing issue for our recent migration from J3 latest to J4.2.3.

I am also willing to test, but the following link seems to be dead: https://ci.joomla.org/artifacts/joomla/joomla-cms/4.2-dev/37962/downloads/58382/

Also I need to get setup properly in docker first, perhaps sometime this week pending time available.

What I am seeing is an issue with a connection to the Symphony adapter, if that is immediately helpful: 2022-10-25T23:01:28+00:00 CRITICAL 172.19.1.212 error Uncaught Throwable of type TypeError thrown with message "ldap_set_option(): Argument #1 ($ldap) must be of type ?LDAP\Connection, bool given". Stack trace: #0 [ROOT]/libraries/vendor/symfony/ldap/Adapter/ExtLdap/Connection.php(104): ldap_set_option()

I have tried the fix suggested here, which doesn't work for me: https://github.com/joomla/joomla-cms/issues/35829

Is this still the most direct/recent tree to test from? https://github.com/tatankat/joomla-cms/tree/patched

As I stated, I'm willing to test, or even help with development.

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/37962.}

[Hackwar]

@noxidsoft I restarted the build. In a few minutes a new link to the download package will appear here, which will work.

[noxidsoft]

@Hackwar Tested your changes, now get following which is progress. I'm just checking my Map settings are correct with a response dump.

Joomla everything.php log

2022-10-26T07:04:58+00:00	INFO 172.19.1.212	ldapfailure	Username and password do not match or you do not have an account yet.
2022-10-26T07:04:58+00:00	WARNING 172.19.1.212	jerror	Username and password do not match or you do not have an account yet.