Memory leak in deqMany

[boriborm]

1. What versions are you using?

Platform: linux Node version: v14.17.6 Architecture: x64 node-oracledb version: 5.5.0 Oracle client version: 21.3.0.0.0

2. Is it an error or a hang or a crash?

Crash

3. What error(s) or behavior you are seeing?

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory

4. Include a runnable Node.js script that shows the problem.

const oracledb = require('oracledb');
const conf = require('./config.json');
async function runQueue () {
  const pool = await oracledb.createPool(conf); // in conf connectString, user, password, events=true
  const conn = await pool.getConnection();
  const queue = await conn.getQueue('XXI.CLIENT_OUT');
  queue.deqOptions.visibility = oracledb.AQ_VISIBILITY_ON_COMMIT;
  queue.deqOptions.wait = 0;

  let c = 0;
  console.log(`start`);
  while (c<2000){
    await queue.deqMany(1000);  //queue hasn't messages, return undefined. More maxMessages value -> larger RSS leak
    const stats = process.memoryUsage ();
    console.log ( `count: ${c}, rss: ${stats.rss} `) ;
    c++;
  }
  console.log(`end`);
  await conn.close();
  await pool.close(1);
}
runQueue().catch(err=>console.log(err));

output:

start
count: 0, rss: 71262208 
count: 1, rss: 71585792 
count: 2, rss: 71585792 
count: 3, rss: 71852032 
...
...
count: 1989, rss: 203001856 
count: 1990, rss: 203001856 
count: 1991, rss: 203272192 
count: 1992, rss: 203542528 
count: 1993, rss: 203812864 
count: 1994, rss: 204083200 
count: 1995, rss: 204083200 
count: 1996, rss: 204083200 
count: 1997, rss: 204083200 
count: 1998, rss: 204083200 
count: 1999, rss: 204083200 
end

[cjbj]

Can you add code to create the queue and add runnable code do the enqueues, so we know what's in the queue?

[boriborm]

Queue is empty.

BEGIN 
  DBMS_AQADM.CREATE_QUEUE_TABLE(queue_table => 'MSG_QT',queue_payload_type => 'RAW'); 
  DBMS_AQADM.CREATE_QUEUE(queue_table => 'MSG_QT', queue_name => 'CLIENT_OUT);
  DBMS_AQADM.START_QUEUE('CLIENT_OUT); 
END

[cjbj]

@pvenkatraman another task for you :)

[boriborm]

Maybe bug in function dpiQueue_deqMany

When it run, then numMsgProps changed to 0 (no new messages, queue is empty)

I run code below and no memory leaks

static bool njsAqQueue_deqManyAsync(njsBaton *baton)
{
    njsAqQueue *queue = (njsAqQueue*) baton->callingInstance;

    baton->msgProps = calloc(baton->numMsgProps, sizeof(dpiMsgProps*));

    if (!baton->msgProps)
        return njsBaton_setError(baton, errInsufficientMemory);
/*
    if (dpiQueue_deqMany(queue->handle, &baton->numMsgProps,
            baton->msgProps) < 0)
        return njsBaton_setErrorDPI(baton);
*/
    baton->numMsgProps = 0; // Empty queue imitation
    return true;
}

Script in my first post output:

start
count: 0, rss: 60858368 
count: 1, rss: 61128704 
count: 2, rss: 61394944 
count: 3, rss: 61394944 
count: 4, rss: 61394944 
count: 5, rss: 61394944 
count: 6, rss: 61394944 
count: 7, rss: 61394944 
...
...
count: 1994, rss: 61669376 
count: 1995, rss: 61669376 
count: 1996, rss: 61669376 
count: 1997, rss: 61669376 
count: 1998, rss: 61669376 
count: 1999, rss: 61669376 
end

memory usage between 60858368 and 62414848

[pvenkatraman]

Thanks @boriborm will take a look.

[boriborm]

Bug in dpiQueue__deq when numMsgProps > 1 and payload is 'RAW', then run this function

status = dpiOci__aqDeqArray(queue->conn, queue->name,
                queue->deqOptions->handle, numProps, queue->buffer.handles,
                payloadTDO, queue->buffer.instances, queue->buffer.indicators,
                queue->buffer.msgIds, error);

This function run OCI function OCIAQDeqArray :(

When i change it to

   /*
        status = dpiOci__aqDeqArray(queue->conn, queue->name,
                queue->deqOptions->handle, numProps, queue->buffer.handles,
                payloadTDO, queue->buffer.instances, queue->buffer.indicators,
                queue->buffer.msgIds, error);
  */
  status = 0;
  *numProps = 0; // Empty queue imitation

no memory leak

[pvenkatraman]

Thanks, working with another team member on this issue. Will update soon.

[JoshuaCEnrico]

Is this problem solved in any specific version? Version less than 6 from here

[sharadraju]

@JoshuaCEnrico The memory profile with node-oracledb 6.x has changed slightly, because of the refactoring to use less C code in general, in addition to the newly introduced Thin mode. We will try to see if this can be reproduced in the 6.x version and get back to you.

[999max]

@sharadraju The issue still exist, even in the v6.5.1

[sharadraju]

@999max There were some memory leak issues with AQ which were fixed in the latest Oracle Instant Client libraries. Can you please try your test case with the latest Oracle Instant Client libraries: https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html

[999max]

hey @sharadraju
I tested with v6.6.0 but unfortunately the issue with memory leak in deqMany was NOT fixed

[sharadraju]

@999max Can you please indicate what Oracle Instant Client library version was used. You will have to use the latest Oracle Instant Client libraries, preferably Oracle Instant Client 23.5. I am assuming the client platform is Linux.

[sosoba]

preferably Oracle Instant Client 23.5

Hi @sharadraju . Is this an official recommendation? Our DBA suggests taking a conservative approach and using 19 client for 19 database etc.

[cjbj]

The issue we think it was was fixed in 23.5. I don't see a backport for 19 having landed yet, but it was requested. The Oracle bug is 36741214

[999max]

@sharadraju Sorry, I didn't understand your question reagrding the 'Oracle Instant Client'. I guess I don't use the 'Oracle Instant Client'. I'm using only the 'node-oracledb' package inside my NestJs app for queue reading (deqMany, deqOne). And, as mentioned above, there is a memory leak in deqMany() only.

[boriborm]

@999max , Проблема не в node-oracledb. Проблема в oracle instant client, который использует node-oracledb. В 6-й версии, насколько я помню, работа с очередями средствами node-oracledb напрямую с сервером ещё не поддерживается и работает только через использование оракл клиент. А ошибка в нем. И соответствующий запрос уже в оракл сделан. Но там эту проблему вероятно решили в версии оракл клиента 23.5. Я дальше 21-й версии не проверял, т.к. реализовал код используя deqOne.