Possibility to test error scenarios

[MichalCho]

Hi! I was wondering if it is possible to test ldap busy scenarios? (Sorry if I missed that info in the documentation).

Parameters that I used for starting the server: unboundid-ldapsdk-se.jar com.unboundid.ldap.listener.InMemoryDirectoryServerTool --baseDN=dc=com --port=1389 --ldifFile=/tmp/ldapjHKOmH/ldap.ldif

Thank you a lot for your answer.

[dirmgr]

It sounds like you want the server to return a response of "busy" to all requests. You can't do that from the command line, but it's very easy to do programmatically. Here's an example:

/*
 * Copyright 2018 Ping Identity Corporation
 * All Rights Reserved.
 */
/*
 * Copyright (C) 2018 Ping Identity Corporation
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License (GPLv2 only)
 * or the terms of the GNU Lesser General Public License (LGPLv2.1 only)
 * as published by the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, see <http://www.gnu.org/licenses>.
 */
import java.util.List;

import com.unboundid.ldap.listener.CannedResponseRequestHandler;
import com.unboundid.ldap.listener.LDAPListener;
import com.unboundid.ldap.listener.LDAPListenerConfig;
import com.unboundid.ldap.sdk.Filter;
import com.unboundid.ldap.sdk.LDAPConnection;
import com.unboundid.ldap.sdk.LDAPResult;
import com.unboundid.ldap.sdk.ResultCode;
import com.unboundid.ldap.sdk.SearchRequest;
import com.unboundid.ldap.sdk.SearchScope;
import com.unboundid.util.LDAPTestUtils;



/**
 * This class demonstrates how to use the LDAP SDK to create an LDAP listener
 * that always returns a canned response with a result code of busy (51) to any
 * request that it receives.
 */
public final class CannedResponseLDAPListenerExample
{
  /**
   * Runs this program with the provided set of arguments.
   *
   * @param  args  The command-line arguments provided to this
   *               program.  Any arguments given will be ignored.
   *
   * @throws  Exception  If an unexpected problem occurs.
   */
  public static void main(final String... args)
         throws Exception
  {
    // Define all the parameters that will be used for this example.
    final int listenPort = 1389;
    final ResultCode cannedResultCode = ResultCode.BUSY;
    final String cannedMatchedDN = null;
    final String cannedDiagnosticMessage =
         "The server is too busy to process your request";
    final List<String> cannedReferralURLs = null;


    // Create an LDAP listener that will always return a canned
    // response to any request.
System.out.println("Creating the LDAP listener...");
    final LDAPListenerConfig listenerConfig =
         new LDAPListenerConfig(listenPort,
              new CannedResponseRequestHandler(cannedResultCode,
                   cannedMatchedDN, cannedDiagnosticMessage,
                   cannedReferralURLs));

    final LDAPListener listener = new LDAPListener(listenerConfig);
    listener.startListening();
System.out.println("Listening on port " + listenPort);


    // Establish a connection to the LDAP listener and send a search
    // request.  Verify that we get a response with the expected
    // result code.
    try (final LDAPConnection conn =
              new LDAPConnection("localhost", listenPort))
    {
System.out.println("Successfully established a connection");
      final SearchRequest searchRequest = new SearchRequest(
           "dc=example,dc=com", SearchScope.BASE,
           Filter.createPresenceFilter("objectClass"));
System.out.println("Sending search request " + searchRequest);

      final LDAPResult searchResult =
           LDAPTestUtils.assertResultCodeEquals(conn,
                searchRequest, cannedResultCode);
System.err.println("Got search result " + searchResult);
    }
    finally
    {
      listener.shutDown(true);
    }
  }
}

[MichalCho]

Hi,

Thank you for your answer, I will check it out and come back with feedback but I guess this is what I need. By the way I was really surprised how detailed your response was. I really appreciate that.

I have another question related to pooling, I will post it here but if you prefer I can create a separate thread.

What we need in our application is a pool with a maximum number of connection (what we can get now). The requirement is that an idle connection are removed from the pool after a specified period (what we can do now with ldapConnectionPool.setMaxConnectionAgeMillis(connection_lifetime);)

However I've noticed that even I set ldapConnectionPool.setMinimumAvailableConnectionGoal(0); There is always one connection established between our application and ldap server. Althought it's closed and reopend every HealthCheckInterval. Is it possible to have a pool without any connection active if an application is idle for some time.

Below you can find how I create a pool:

        ldapConnection = new LDAPConnection("localhost",1389 );

        ldapConnectionPool = new LDAPConnectionPool(ldapConnection, 10);
        ldapConnectionPool.setMaxConnectionAgeMillis(1l);
        ldapConnectionPool.setHealthCheckIntervalMillis(1000);
        ldapConnectionPool.setMinimumAvailableConnectionGoal(0);
        ldapConnectionPool.setCreateIfNecessary(false);
        ldapConnectionPool.setMaxWaitTimeMillis(1000);

Thank you a lot for your answer. Best regards, Michal

Edit: What's more I've seen that opened connections and health check thread run forever, even if I stop my application. I guess it is not correct. Br, Michal

[dirmgr]

The LDAP SDK doesn’t currently provide a way to automatically reduce the number of connections in a pool after a specified period of time. That really isn’t a recommended practice because it’s much cheaper to keep a connection around even if it’s no longer needed than to need a connection when one isn’t available so you either need to block or create a new one.

It sounds like you’ve got a misunderstanding about what the minimum available connection goal is. It doesn’t have anything to do with shrinking the pool, and it doesn’t close or discard connections. Rather, it is used to help ensure that the pool always has enough connections available in case there’s a sudden surge of new connections. The pool will periodically monitor itself (at the health check interval) and see how many connections are available for immediate use. If that number is less than the minimum available connection goal, then the pool will create enough additional connections to reach that goal.

At present, there are only two ways that you can reduce the number of connections in a pool:

• Check out a connection and instead of using releaseConnection to put it back, you use discardConnection.

• Use the shrinkPool method to specify the number of connections you want to retain, and if the pool has more than that number of connections, it will discard enough connections to reach that minimum. This is really just a convenience wrapper that checks out connection and calls discardConnection enough times to reduce the size of the pool to the desired number of connections.

I would generally not recommend trying to reduce the size of the pool unless you’re sure that the extra connections won’t be needed because it’s much better to have them available and the cost of maintaining them is negligible. However, enough people have asked about this feature that I’ll consider adding some kind of automated way to reduce the number of connections if they haven’t been needed for an extended period of time.

And to address your comment about seeing the connections established and the health check thread running after your application is stopped: that sounds like you didn’t close the connection pool when stopping your application. If you don’t close the pool, the connections will remain established and the health check thread will stay running.

[dirmgr]

I just committed a change to the LDAP SDK that added a new PruneUnneededConnectionsLDAPConnectionPoolHealthCheck class that can be used to periodically monitor the size of the pool (at the pool’s health check interval). If the pool consistently has more than a specified minimum number of available connections for longer than a given length of time, then the health check can shrink the pool to that minimum.

[MichalCho]

Hi Neil,

again thank you a lot for your super fast answer. I will check it out tomorrow. The reason behind my question is a requirement from our customer to not keep idle connection open for a long time. Firstly we've tried to used JNDI and it provides this kind of mechanism (connections that have been in idle state longer that specified interval, are removed from the pool during next health check). As our application is expected to run 24/7 with many instances, keeping all the connection open all the time may lead to occupy all the connections available on the server side, if I understood correctly how the ldap server works.

Regarding the second issue - yes, you're right. I have not closed the pool. I was testing a scenario when user decided to abort a task executed by a worker. My impression was that then a pool object will be removed during next GC and all the connections will be closed prior to that. Is there a way to handle this kind of scenario so we don't end up with growing number of opened connection?

I really appreciate your help and will be looking forward to your answer. Michal

[dirmgr]

LDAP connections and connection pools do have finalize methods that do ensure that they’re closed whenever that method gets run. However, finalizers are highly unpredictable and it can be a very long time between the time that an object falls out of scope and its finalizer is called, and there may be times when it never gets called at all.

The finalize method has been deprecated in Java 9, where they recommend that you use java.lang.ref.Cleaner instead, but the LDAP SDK can’t make use of that because it currently needs to support Java 7 and later. If your application runs in Java 9 or later, then that’s the way to go.

Alternately, your application might also provide some other way of invoking code when it gets shut down. For example, if the application runs as a servlet, you can put a call to close the connection pool in the destroy method.

[MichalCho]

ok, I get it. Unfortunately we can not use just Java 9 approach as we have to support Java 7 and 8.

Regarding the finalize method - I've seen it in the documentation so my idea was that it would be called during next GC, but true, I can not count on that (even tough I've triggered several GC hoping it will take effect, I've also left our application running for some time - still connection were opened).

Anyway let me try PruneUnneededConnectionsLDAPConnectionPoolHealthCheck and I will come back to you with feedback.

Once again, thank you for help. Best regards, Michal