AdsiPS icon indicating copy to clipboard operation
AdsiPS copied to clipboard

Performance - Recommended approach when query AD

Open lazywinadmin opened this issue 9 years ago • 11 comments

If you take a look at the functions from this module, you'll see that I mostly use the namespace system.directoryservices.activedirectory mostly because the object returned already give some nice properties and methods that I can use to use to perform other actions.

However in previous version of most functions, I was using the System.DirectoryServices.DirectorySearcher (You can take a look in the Archive folder) and the queries were much much faster. The only inconvenient with this one is...that I have to do a lot of work to show each properties and I don't get any methods unfortunately.

Maybe I'm seeing a problem that is not one, and this can be fixed very easily,

Any advices ?

http://stackoverflow.com/questions/23176284/difference-between-principlesearcher-and-directorysearcher

cc: @evetsleep

lazywinadmin avatar Aug 18 '16 02:08 lazywinadmin

Sorry for taking so long to reply.

So it would seem there is a balance to be struck. One one hand you have an approach which provides a little less control, but adds simplicity (at the cost of performance), but on the other hand you have an alternative approach which can be complex, but gives you a greater range of control and, overall, is faster. For example, the Get-ADSIGroupMember function is significantly slower compared to using just System.DirectoryServices.DirectorySearcher (S.DS.DS) to get group memberships. I have a group with 575 members and it took S.DS.DS (across a VPN and 3,000 miles away from the DC) ~300 milliseconds to pull down the members. Conversely using System.DirectoryServices.AccountManagement.GroupPrincipal (S.DS.AM.GP) to get the same groups members took over 2 minutes. The difference of course is the data you get back. With S.DS.DS you're only getting the membership DN's, but with S.DS.AM.GP you're getting full member detail. Maybe it might be worth it to make getting member detail optional in the interest of speed?

I have another group (because I wanted to see if S.DS.AM.GP would page large group memberships...it does) of over 5,000 members and it takes S.DS.DS ~800 milliseconds to pull down the memberships. S.DS.AM.GP took so long that I went and made breakfast and came back and it was still churning :).

For me I tend to prefer high performance over simplicity because I work in an environment with a large number of users and groups and many of my tools pull large numbers of them down from AD for various purposes. 99.9999% of the time I don't get group memberships for the purpose of learning about the members themselves. I just need their DN's.

It's definitely a problem, but only if you're married to using S.DS.AM.GP. I can still do the same things with S.DS.DS in most cases but faster (but as you pointed out, with a bit more code).

Cc: @lazywinadmin

evetsleep avatar Aug 23 '16 11:08 evetsleep

Thanks @evetsleep for the great info. Yeah it make sense.

Can we select the properties to be returned ? PropertiesToLoad.Add() PropertiesToLoad.AddRange()

I did not test yet, but it appears to do what we want

Thanks again

lazywinadmin avatar Aug 23 '16 14:08 lazywinadmin

With S.DS.DS Yes!!! When adding properties to load it will significantly improved the time it takes to pull down data from Active Directory. This is especially true for large queries. For example, if I query for everyone whose name starts with S where I work and don't specify what properties to return, then among other things I'll get back all their member properties (which average about 200 groups per user) as well as published certificate data in Active Directory and Unified Messaging data. For one user it's fine, but for thousands it starts to add up and becomes almost useless.

I can write up an example tonight if it'll help.

On Tue, Aug 23, 2016 at 10:21 AM, François-Xavier Cat < [email protected]> wrote:

Thanks @evetsleep https://github.com/evetsleep for the great info. Yeah it make sense.

Can we select the properties to be returned ? PropertiesToLoad.Add() PropertiesToLoad.AddRange()

I did not test yet, but it appears to do what we want

Thanks again

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lazywinadmin/AdsiPS/issues/8#issuecomment-241746980, or mute the thread https://github.com/notifications/unsubscribe-auth/AGBoYGcp9rR9wDZ-q3GGX0b5Jt5WZgluks5qiwHugaJpZM4JnENZ .

evetsleep avatar Aug 23 '16 15:08 evetsleep

@evetsleep That would be nice, Will check on my side too :-) btw are you on twitter or something ?

lazywinadmin avatar Aug 23 '16 23:08 lazywinadmin

I'm not as active as I should probably be, but yeah:

Twitter: @evetsleep Mail: [email protected] Github: http://evetsleep.github.io/

On Tue, Aug 23, 2016 at 7:07 PM, François-Xavier Cat < [email protected]> wrote:

That would be nice, Will check on my side too :-) btw are you on twitter or something ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lazywinadmin/AdsiPS/issues/8#issuecomment-241908358, or mute the thread https://github.com/notifications/unsubscribe-auth/AGBoYOCvr-zsOxP4subuHnmqD241NVkDks5qi301gaJpZM4JnENZ .

evetsleep avatar Aug 23 '16 23:08 evetsleep

@evetsleep Nice, added you.

This article summarize well the approach I took, the 'new way'. http://ianatkinson.net/computing/adcsharp.htm

I looked at adding PropertiesToLoad but it seems only available with DirectorySearcher. Not sure how to improve the query speed when using S.DS.AM.GP

lazywinadmin avatar Aug 24 '16 15:08 lazywinadmin

That's a pretty interesting article that I haven't seen before thanks for sharing. The author does a good job breaking down the two approaches, however is very focused on single uses. If they would look at a larger picture I think S.DS.AM becomes a little less attractive. I think part of the problem with S.DS.AM is that it doesn't give the flexibility to do something like PropertiesToLoad...it just gives you everything (including some useful methods as you've mentioned, but nothing that cannot be addressed with a little effort). In small\medium operations that's ok, but it doesn't scale well.

So I think that it comes down to design choices. For speed and flexibility the cost is complexity. I don't think there is anything wrong with using S.DS.AM honestly, just need to be aware of its limitations. I have some processes which handle thousands of groups, computers, and users per execution and that means something like S.DS.AM won't really work for me. I think the power of S.DS.AM comes in single operations or small directory queries.

On Wed, Aug 24, 2016 at 11:17 AM, François-Xavier Cat < [email protected]> wrote:

@evetsleep https://github.com/evetsleep Nice, added you.

This article summarize well the approach I took, the 'new way'. http://ianatkinson.net/computing/adcsharp.htm

I looked at adding PropertiesToLoad but it seems only available with DirectorySearcher. Not sure how to improve the query speed when using S.DS.AM.GP

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lazywinadmin/AdsiPS/issues/8#issuecomment-242101764, or mute the thread https://github.com/notifications/unsubscribe-auth/AGBoYH7WDSFDv8-OHBdllUDrP9T8ZZyxks5qjGCSgaJpZM4JnENZ .

evetsleep avatar Aug 24 '16 15:08 evetsleep

@evetsleep

What about using... PrincipalSearcher ? We can access the DirectorySearcher Looks pretty fast

function Test-PrincipalSearcher
{
# Principal Context
$PrincipalContext = New-Object -TypeName System.DirectoryServices.AccountManagement.PrincipalContext -ArgumentList $([System.DirectoryServices.AccountManagement.ContextType]::Domain),$([System.DirectoryServices.ActiveDirectory.Domain]::Getcurrentdomain())
# GroupPrincipal with PrincipalContext
$GroupPrincipal = New-object -type System.DirectoryServices.AccountManagement.GroupPrincipal -ArgumentList $PrincipalContext
# PrincipalSearcher
$PrincipalSearcher = new-object -TypeName System.DirectoryServices.AccountManagement.PrincipalSearcher -ArgumentList $GroupPrincipal

# Get the DirectorySearcher object !!
# $PrincipalSearcher.GetUnderlyingSearcher()


# Clear the properties to load
[void]$PrincipalSearcher.GetUnderlyingSearcher().propertiestoload.clear()
[void]$PrincipalSearcher.GetUnderlyingSearcher().propertiestoload.add('sAMAccountName')
#[void]$PrincipalSearcher.GetUnderlyingSearcher().propertiestoload.addrange('samaccountname','distinguishedname')

# Even after the clear, still have 12 properties
# $PrincipalSearcher.GetUnderlyingSearcher().propertiestoload

# Find methods available on propertiestoload
# $PrincipalSearcher.GetUnderlyingSearcher().propertiestoload.psbase|gm -force

foreach ($i in $PrincipalSearcher.FindAll())
{
    $CurrentObject = $i.GetUnderlyingObject() #|gm 
    $CurrentObject.samaccountname
    #$CurrentObject.distinguishedname
}
}
Test-PrincipalSearcher

lazywinadmin avatar Aug 24 '16 21:08 lazywinadmin

@evetsleep

Actually, I was not talking to the DirectorySearcher directly, here is a corrected version

function Test-PrincipalSearcher
{
    # Principal Context
    $PrincipalContext = New-Object -TypeName System.DirectoryServices.AccountManagement.PrincipalContext -ArgumentList $([System.DirectoryServices.AccountManagement.ContextType]::Domain), $([System.DirectoryServices.ActiveDirectory.Domain]::Getcurrentdomain())
    # GroupPrincipal with PrincipalContext
    $GroupPrincipal = New-object -type System.DirectoryServices.AccountManagement.GroupPrincipal -ArgumentList $PrincipalContext
    # PrincipalSearcher
    $PrincipalSearcher = new-object -TypeName System.DirectoryServices.AccountManagement.PrincipalSearcher -ArgumentList $GroupPrincipal

    # Get the DirectorySearcher object !!
    # $PrincipalSearcher.GetUnderlyingSearcher()

    $DirectorySearcher = $PrincipalSearcher.GetUnderlyingSearcher()

    # Clear the properties to load
    [void]$DirectorySearcher.propertiestoload.clear()
    [void]$DirectorySearcher.propertiestoload.add('sAMAccountName')
    #[void]$DirectorySearcher.propertiestoload.addrange('samaccountname','distinguishedname')

    # Even after the clear, 1 property to load
    # $PrincipalSearcher.GetUnderlyingSearcher().propertiestoload

    foreach ($i in $DirectorySearcher.FindAll())
    {
        #$CurrentObject = $i.GetUnderlyingObject() #|gm
        $i.properties['samaccountname']
    }
}
Test-PrincipalSearcher

lazywinadmin avatar Aug 24 '16 21:08 lazywinadmin

Related: https://evetsleep.github.io/adsi/2017/02/22/SDSFormat.html

lazywinadmin avatar Jun 14 '17 14:06 lazywinadmin