gapic-generator-typescript icon indicating copy to clipboard operation
gapic-generator-typescript copied to clipboard

NodeJS generator seems to generate clients with memory leak

Open surbhigarg92 opened this issue 1 year ago • 2 comments

The NodeJS gapic clients seem to have a memory leak that can be triggered using the following steps:

Open a gapic client. Execute one simple request. Close the gapic client. Repeat.

This issue was reported for NodeJS Spanner client library. Internal customer was trying to investigate memory leak issue in Spanner Autoscaler which led them to discover memory leaks in NodeJS Spanner client. Please refer issue . When investigated this further it seems to be more of a generic problem with the generated gapic clients.

Running a simple script that opens a simple generated client, executes one request with it and then closes the client will now leak approx 3MB after 100 iterations.

Test Script

const {Spanner, v1} = require('@google-cloud/spanner');
const REPETITIONS = 100;
console.log(`Running ${REPETITIONS} times`);

function main() {
    async function test() {
        for (let i = 0; i < REPETITIONS; i++) {
            await useGapic();
            global.gc();
            const used = process.memoryUsage().heapUsed / 1024 / 1024;
            console.log(`${i}: Current mem usage ${Math.round(used * 100) / 100} MB`);
        }
    }

    async function useGapic() {
        const client = new v1.InstanceAdminClient({
            projectId: 'my-project',
            keyFile: '/path/to/my-key.json',
        });
        await client.listInstanceConfigs({
            parent: 'projects/my-project',
        });
        await client.close();
    }
    test();
}

process.on('unhandledRejection', err => {
    console.error(err.message);
    process.exitCode = 1;
});
main();

This is continuation to what was also reported in bug

Thanks

surbhigarg92 avatar Mar 07 '23 04:03 surbhigarg92

I can repro the leak, looking.

$ node --expose-gc index.js 
Running 100 times
0: Current mem usage 23.25 MB
1: Current mem usage 23.4 MB
2: Current mem usage 23.66 MB
3: Current mem usage 23.77 MB
4: Current mem usage 23.95 MB
5: Current mem usage 24.08 MB
6: Current mem usage 24.17 MB
7: Current mem usage 24.24 MB
8: Current mem usage 24.44 MB
9: Current mem usage 24.57 MB
10: Current mem usage 24.66 MB
11: Current mem usage 24.75 MB
12: Current mem usage 24.82 MB

alexander-fenster avatar Mar 10 '23 20:03 alexander-fenster

Reported to gRPC ↑↑↑

alexander-fenster avatar Mar 10 '23 21:03 alexander-fenster