size limit exceeded

[TamasSzerb]

Hello,

With Microsoft Active Directory occuring:

2016-09-19 11:29:46 [INFO ] Start fetching empty teams from ldap. 2016-09-19 11:29:47 [ERROR] Problem Searching LDAP LDAPSearchException(resultCode=4 (size limit exceeded), numEntries=1000, numReferences=0, errorMessage='size limit exceeded') at com.unboundid.ldap.sdk.LDAPConnection.search(LDAPConnection.java:3657) at com.gitblit.auth.LdapAuthProvider.doSearch(LdapAuthProvider.java:549) at com.gitblit.auth.LdapAuthProvider.getEmptyTeamsFromLdap(LdapAuthProvider.java:504) at com.gitblit.auth.LdapAuthProvider.sync(LdapAuthProvider.java:173) at com.gitblit.service.LdapSyncService.run(LdapSyncService.java:55) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2016-09-19 11:29:47 [INFO ] Finished fetching empty teams from ldap. 2016-09-19 11:29:47 [INFO ] Finished user and group sync with ldap service

So I think that the paging should be used b/c MSAD in this example limits the response to one thousand items.

Thanks,

Tamas

[TamasSzerb]

eg. https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/controls/SimplePagedResultsControl.html or https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/controls/VirtualListViewRequestControl.html

[gitblit]

It's a good idea. Would you be interested in implementing it?

[flaix]

Is it necessary to have all these (empty) teams in Gitblit? Could this also be solved if there was a way to better filter the teams that are synchronized into Gitblit from LDAP?

[TamasSzerb]

I wonder about different approach. Now it queries the users by realm.ldap.accountPattern, it can be possible to check the groups by reading the memberOf attributes of the user which provides the groups' DN instead of the realm.ldap.groupMemberPattern. I find this definitely more elegant (if the given LDAP server supports it, the MS AD does), and more efficient because it does require to iterate over the all of groups. Opinion?