LDAP Group Searches Not Paging

[asummers]

If you are reporting a problem, please make sure the following information are provided:

Expected behavior and actual behavior: It is very common to have lots of groups in an organization of even moderate size, and it seems Harbor is only pulling the first 2500 groups (AD page limit). Running same LDAP query using tools that page return many more results. As group result ordering is not well defined in AD, this manifests in inconsistent group synchronization for users. More frustratingly, if a group was already synced, then was manipulated after there were more groups added (pushing it off first page), then membership will not be synced and lead to some users having access and some not, depending on when it was synced.

I believe it should eventually be calling https://github.com/goharbor/harbor/blob/44f477e965e57ba619d494a7122c69a636906666/src/vendor/github.com/go-ldap/ldap/v3/search.go#L293 instead but it is not.

Steps to reproduce the problem:

1. Create LDAP/AD that has >2500 groups
2. Set LDAP search query in config
3. Note that group count in Groups section of nav only has 2500 results.

In my case I expanded the search query to widen it past 2500, so that may contribute to some of the inconsistencies I am seeing, but something is definitely afoor

Versions: Please specify the versions of following systems.

• harbor version: [x.x.x] Version v2.3.0-047b122c
• docker engine version: [y.y.y] N/A
• docker-compose version: [z.z.z] N/A

Additional context:

• Harbor config files: You can get them by packaging harbor.yml and files in the same directory, including subdirectory.
• Log files: You can get them by package the /var/log/harbor/ . N/A

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[asummers]

This is still an issue that impacts us. Please do not close.

[stonezdj]

@asummers, the user group in the nav doesn't retrieve from LDAP/AD server, they are LDAP group which are sync to Harbor database when user login, the searchWithPaging deosn't impact the user group query in navi. I agreed to replace the ldap.Search with ldap.SearchWithPaging, but it is not relate to the issue you mentioned.

[jolt23]

I am not sure if this could be a limitation to what we started to see. In our Harbor we had a working configuration for some months but some time over the weekend we had members who could no longer see certain projects. I wonder if there is some type of limitation on number of LDAP groups that Harbor can handle.

We temporarily fixed the issue by narrowing down the LDAP based dn search from dc=xxx,dc=xxx,dc=xxx to ou=xxx,dc=xxx,dc=xxx,dc=xxx. Only issue with this is our organization has groups across multiple OU and we are having to manually add those users to projects now. I don't all the information to create a new ticket on this issue but wanted to ask if there are some limitations to number of groups either a user can be member of or number of groups returned by a group search?

[jolt23]

I have an update on my comment. I crafted a query for LDAP that narrows down our LDAP Group Filter and that seems to have allowed us to keep our Base LDAP DN as dc=xxx,dc=xxx,dc=xxx, I imagine this points to some limitation in the number of groups returned that Harbor handles. I don't know if this is a bug or not, or maybe it should just be identified in documentation. I am going to keep our improved filter query as it should perform better and narrow down to the groups we need.