async-local-storage icon indicating copy to clipboard operation
async-local-storage copied to clipboard

Should it be destoyed when Promise.resolve is invoked?

Open cmershen1 opened this issue 5 years ago • 15 comments

when the resolve function is invoked, the promiseResolve callback function will be called. I used your library in my project, and got memory leak in the end. I think the reason is promiseResolve isn't handled. Am I right?

cmershen1 avatar May 13 '19 08:05 cmershen1

Promises are destroyed by gc, so the data of promise will be removed later, which is similar to the situation you mention. I raise a example below, if your problem hasn't been solved, please show me you code.

node --expose-gc test.js
const als = require('async-local-storage');

als.enable();

setTimeout(() => {
  new Promise((resolve) => {
    als.set("a", 1);
    resolve();
  }).then(() => {
    als.set("b", 1);
    // getAllData is supported in version 2.2.0
    console.dir(als.getAllData());
    return Promise.resolve();
  })
})

setTimeout(() => {
  console.dir(als.getAllData());
  global.gc();
}, 1000)
setTimeout(() => {
  console.dir(als.getAllData());
}, 2000);

vicanso avatar May 13 '19 13:05 vicanso

@cmershen1 Has the problem you mentioned been solved?

vicanso avatar May 20 '19 12:05 vicanso

@cmershen1 Has the problem you mentioned been solved?

my question has not been solved yet. I checked my code again and again, can this usage cause memory leak?

exports.query = function(sql, data=[]) {
    const traceid = ALS.get('__traceid');
    const start = ALS.get('__start');

	return new Promise((resolve, reject)=>{
		pool.query(sql, data, (err, results, fields) => {
            ALS.set('__traceid', traceid);
            ALS.set('__start', start);
                        
			if (err) {
				reject(err);
			} else {
				resolve(results);
			}
		});
	});
}

cmershen1 avatar May 22 '19 08:05 cmershen1

I tried to print the map, the output is like this: 2019-05-22 17:21:13|25634|INFO|app.js:40|1184,{"created":[1558516873,924536775],"parent":{"created":[1558516871,925288429],"parent":{"created":[1558516869,924660300],"parent":{"created":[1558516867,924269489],"parent":{"created":[1558516865,923400076],"parent":{"created":[1558516863,922600607],"parent":{"created":[1558516861,922927699],"parent":{"created":[1558516859,922768321],"parent":{"created":[1558516857,921752524],"parent":{"created":[1558516855,920830227],"parent":{"created":[1558516853,921297443],"parent":{"created":[1558516851,921274300],"parent":{"created":[1558516849,920369574],"parent":{"created":[1558516847,919384971],"parent":{"created":[1558516845,918546701],"parent":{"created":[1558516843,918050732],"parent":{"created":[1558516841,918103027],"parent":{"created":[1558516839,917946609],"parent":{"created":[1558516837,917317481],"parent":{"created":[1558516835,917083043],"parent":{"created":[1558516833,917044200],"parent":{"created":[1558516831,914159696],"parent":{"created":[1558516829,913760412],"parent":{"created":[1558516827,912996596],"parent":{"created":[1558516825,912413567],"parent":{"created":[1558516823,912186882],"parent":{"created":[1558516821,912223413],"parent":{"created":[1558516819,912218556],"parent":{"created":[1558516817,912167310],"parent":{"created":[1558516815,911406521]}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}

and the layer becomes more and more deeper.

cmershen1 avatar May 22 '19 09:05 cmershen1

It seem abnormal and I found that each item of data you shown 2ms deviated. Did you use setInterval ?

Can you please show me complete and operational codes, otherwise I can't help you.

vicanso avatar May 22 '19 14:05 vicanso

It seem abnormal and I found that each item of data you shown 2ms deviated. Did you use setInterval ?

Can you please show me complete and operational codes, otherwise I can't help you.

my server runs on the rpc framework Tars, and my business code doesn't use setInterval anywhere. whatever, I don't know the details on how the rpc framework works clearly.

maybe we can find some clues in : https://github.com/tars-node/rpc https://github.com/tars-node/Tars.js https://github.com/tars-node/monitor

cmershen1 avatar May 23 '19 07:05 cmershen1

You can use scope function to create a new scope.

vicanso avatar May 23 '19 13:05 vicanso

You can use scope function to create a new scope.

how to use scope? I use ALS.scope() in the starting of every RPC function implements. and I use ALS.enable() in app.js when my service starts.

I use scope like the codes below. then I will show my pseudo code as a demo:

app.js:

const FooBarImp = require('FooBarImp');
global.ALS = require('async-local-storage');

ALS.enable();

...
...
createServer(FooBarImp);

FooBarImp.js:

FooBarImp.prototype.function1 = function(req, rsp) {
       ALS.scope();
       ALS.set('traceid', req.traceid);
       ALS.set('start', Date.now());
       // then 业务逻辑 of function1
       sendResponse(rsp);
};

FooBarImp.prototype.function2 = function(req, rsp) {
       ALS.scope();
       ALS.set('traceid', req.traceid);
       ALS.set('start', Date.now());
       // then 业务逻辑 of function2
       sendResponse(rsp);
};

cmershen1 avatar May 23 '19 14:05 cmershen1

Yes, you can use scope function in the starting of each RPC call.

vicanso avatar May 27 '19 13:05 vicanso

@vicanso I have such a problem with the project,have memory leak. you can clone my repository run: 1- npm i 2- npm run dev

then go to '~/src/render/server/server.js' and check my code.

if you run 'als.map()' see in each time map It gets bigger. if you write 'console.log(req.url)' see only one time this function run. search 'als.set()' and 'als.get()' over project to see usage.

ebrahimiaval avatar Mar 09 '20 20:03 ebrahimiaval

Try to call gc like this commemt.

vicanso avatar Mar 10 '20 12:03 vicanso

Try to call gc like this commemt.

if i use GC in high traffic ,am I not mistaken by run gc and cleanup memory between other Express request.

i have another problem. in first run (node v12.10.0, npm 6.11.3) have this error:

(node:896) UnhandledPromiseRejectionWarning: TypeError: Cannot convert undefined or null to object
    at Object.scope (C:\Users\Mohammad\Documents\RSSR\dist\webpack:\node_modules\async-local-storage\als.js:234:1)
    at _callee$ (C:\Users\Mohammad\Documents\RSSR\dist\webpack:\src\render\server\server.js:16:13)

ebrahimiaval avatar Mar 10 '20 15:03 ebrahimiaval

Try to call gc like this commemt.

it has error (node:896) UnhandledPromiseRejectionWarning: TypeError: global.gc is not a function

ebrahimiaval avatar Mar 10 '20 16:03 ebrahimiaval

i back to pass down way and remove async-local-storage because have more memory leak.

ebrahimiaval avatar Mar 10 '20 17:03 ebrahimiaval

The map gets bigger may be not memory leak. Some data will be removed after gc, so you should check the map after gc. If you want to use global.gc, the application should be run like this:

node --expose-gc app.js

vicanso avatar Mar 11 '20 03:03 vicanso