critical memory leak

[yiwiz-sai]

const boa = require('@pipcook/boa');
const { bytes } = boa.builtins();
async function sleep(ms){
        const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
        await delay(ms);
}
async function test(){
        while(1){
                let buf = Buffer.from('f'.repeat(10000000));
                aa = bytes.fromhex(buf.toString('hex'));
                await sleep(1000);
                console.log(aa.slice(0,10));
        }
}
test();

run it in docker, and run "docker stats" to watch the memory.

[yiwiz-sai]

@FeelyChau only want to know if this critical issue will be fixed soon. I am not pushing, but I have a project to use this library, if it need long time to fix. I have to change to other way. Appreciate if you can let me know.

[FeelyChau]

@yiwiz-sai #12 Should fix it, but we still have some problems with testing that need to be fixed. But you could build it from this branch manually.

[yiwiz-sai]

thanks @FeelyChau ! I think this is a very critical bug, so once you finish the fix, will you release a new version to npmjs quickly?

[yiwiz-sai]

@FeelyChau I built and tested, it failed when I import any 3rd party python library

TypeError: ModuleNotFoundError: No module named 'numpy'

At:
  ./detector.py(6): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(728): exec_module
  <frozen importlib._bootstrap>(677): _load_unlocked
  <frozen importlib._bootstrap>(967): _find_and_load_unlocked
  <frozen importlib._bootstrap>(983): _find_and_load

I know that commit failed to pass some tests, as you mentioned.

I think the reason is it has some bugs about module import

[rickycao-qy]

@FeelyChau I built and tested, it failed when I import any 3rd party python library

TypeError: ModuleNotFoundError: No module named 'numpy'

At:
  ./detector.py(6): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(728): exec_module
  <frozen importlib._bootstrap>(677): _load_unlocked
  <frozen importlib._bootstrap>(967): _find_and_load_unlocked
  <frozen importlib._bootstrap>(983): _find_and_load

I know that commit failed to pass some tests, as you mentioned.

I think the reason is it has some bugs about module import

For this one, this is because you will need to use ./tools/bip install numpy to install libraries first. After all, it indeed has some bugs for some kind of import now. We are trying to fix it.

[yiwiz-sai]

@FeelyChau I built and tested, it failed when I import any 3rd party python library

TypeError: ModuleNotFoundError: No module named 'numpy'

At:
  ./detector.py(6): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(728): exec_module
  <frozen importlib._bootstrap>(677): _load_unlocked
  <frozen importlib._bootstrap>(967): _find_and_load_unlocked
  <frozen importlib._bootstrap>(983): _find_and_load

I know that commit failed to pass some tests, as you mentioned. I think the reason is it has some bugs about module import

For this one, this is because you will need to use ./tools/bip install numpy to install libraries first. After all, it indeed has some bugs for some kind of import now. We are trying to fix it.

thanks @rickycao-qy, it works !

[mattseddon]

Would someone be able to approve the workflow run for #15 to see if it fixes the issues with the tests?

Please and thank you.

[mattseddon]

#15 should close this now and maybe #14 too.

[yiwiz-sai]

@rickycao-qy bip can not install python c-extension package, for example

success for common package
node_modules/.bin/bip  install redis

failed for c extension package  (miss Python.h)
node_modules/.bin/bip  install quickjs

hmm, seems ./bin/bip need miniconda, but I can not find miniconda in memory-leak branch

I have to use tools/bip.js to install quickjs