npm-check-updates icon indicating copy to clipboard operation
npm-check-updates copied to clipboard

Incorrectly waits for stdin

Open Igloczek opened this issue 8 years ago • 69 comments

Looks like ncu is currently not compatible with Node v4.1.0. I can run ncu, but nothing happens - cursor just goes to new line.

Igloczek avatar Sep 20 '15 12:09 Igloczek

I installed node v4.1.0 and ran ncu without a problem. All unit tests passing, too.

What happens when you run this command?

echo '{"dependencies":{"express": "1"}}' | ncu

Please provide any additional information you can about the problem.

raineorshine avatar Sep 21 '15 15:09 raineorshine

echo '{"dependencies":{"express": "1"}}' | ncu
express  1  →  4 

ncu without any extra parameters works too, but it takes over 3 minutes to get results...

Igloczek avatar Sep 21 '15 21:09 Igloczek

It looks like it's working. It may take a long time if your package.json is long since npm view has to be called for each dependency to get the latest version number. If you know of a way to get latest versions of multiple dependencies in one call, we could speed this up considerably, but npm view does not appear to support it.

A user experience improvement would be to buffer the results to stdout so that you could see dependencies as they were found, instead of waiting till the end. This would require a significant refactor.

Sorry I don't have a good answer. Let me know if you think of something that I haven't!

raineorshine avatar Sep 22 '15 14:09 raineorshine

I had pretty much the same issue last night, but it was apparently related to slow connection to npm registry. But as you said - it's not really intuitive at the moment and people can just stare there at blinking cursor and wondering that's going on. Is there any options for verbose output?

mrhyde avatar Sep 22 '15 23:09 mrhyde

It would be simple enough to print “Finding latest versions from npm…” at the start.

On Tue, Sep 22, 2015 at 5:46 PM Mr. Hyde [email protected] wrote:

I had pretty much the same issue last night, but it was apparently related to slow connection to npm registry. But as you said - it's not really intuitive at the moment and people can just stare there at blinking cursor and wondering that's going on. Is there any options for verbose output?

— Reply to this email directly or view it on GitHub https://github.com/tjunnone/npm-check-updates/issues/136#issuecomment-142454912 .

raineorshine avatar Sep 23 '15 14:09 raineorshine

Sounds good for me :)

mrhyde avatar Sep 23 '15 15:09 mrhyde

Note I had the same issue, was burning lot so CPU and not returning. Cleaned/removed node_modules directory fixed for me.

kalmanb avatar Sep 24 '15 10:09 kalmanb

@kalmanb Thanks... I'm not sure what is happening there, but definitely is problematic. Let's keep an eye on it, and if we can reproduce it with a given install state then we can debug.

raineorshine avatar Sep 26 '15 17:09 raineorshine

Refactoring so we can check each dependency and print it's state as we get it sounds good to me. It will also make it easier to support interactive mode (#95). For this to work we will need to rethink how we display satisfied dependencies that are not on the latest (currently we display them in a different group after all the unsatisfied).

Mean while, we could use the progress module or something similar, that allows us to show a progress bar in the CLI. The 'total' will be the number of dependencies and as we retrieve the latest version for each one, we will increase it by one. For example:

Finding latest versions [============--------] 60% 13.7s

etiktin avatar Sep 26 '15 21:09 etiktin

Good points. We spent a bit of timing discussing satisfied text, but it's probably worth changing in order to get incremental installs and interactive mode. Let's discuss in #95.

Progress bar would be good!

raineorshine avatar Sep 26 '15 23:09 raineorshine

ncu without any extra parameters works too, but it takes over 3 minutes to get results...

Are you using npm v3? It had pretty bad performance problems until recently (see http://blog.npmjs.org/post/130359991775/npm-weekly-31-npm-3-speed-fixes-modules-demoed and https://github.com/npm/npm/releases/tag/v3.3.6) which could explain the slowness

Daniel15 avatar Oct 11 '15 20:10 Daniel15

@Daniel15 Nope, npm v2.14.4 + node v4.1.1.

Igloczek avatar Oct 12 '15 18:10 Igloczek

I'm tagging this unable-to-reproduce as aside from the performance issues mentioned above, which could give the appearance of nothing happening , I am unable to reproduce a concrete bug.

raineorshine avatar Oct 13 '15 18:10 raineorshine

I'm using npm v2.14.4 and node v4.1.1 with Windows 7, 64bit and ncu is not working. It starts, but never quits. Without any logs I cant check why. Tried to use it with "time ncu", because it may take longer, but I aborted it after 2 hours waiting. My package.json just has 16 entries. Any idea?

mriehema avatar Oct 30 '15 16:10 mriehema

Hi Michael, thanks for reporting. Clearly it is getting hung up somewhere. We should add verbose logs so that we could at least see where it is hanging. Until then, the best we could do is actually debug from the source code.

Does it hang on an empty package.json? echo "{}" > package.json && ncu Does it hang for some of the dependencies but not others? You could test this by removing some dependencies from your package.json and running ncu, or removing all of them and then adding them back in one-by-one.

Adding console.log's or more robust logging is going to be the best approach to really find where in the code it hangs, but since I cannot reproduce you may be on your own for the time being. Perhaps someone else who can reproduce would be willing to debug.

raineorshine avatar Nov 01 '15 23:11 raineorshine

Does it hang on an empty package.json? echo "{}" > package.json && ncu

Yes.

I tried to debug and added some logging to function programRunLocal() before return:

    console.log(pkgFileName);
    console.log(pkgFile);
    console.log(pkgData);

Tested with an empty package.json returns:

    package.json
    undefined
    lib$rsvp$promise$$Promise {
    _id: 0,
    _label: undefined,
    _state: undefined,
    _result: undefined,
    _subscribers: [] }

And it never gets to analyzeProjectDependencies() and doesn't stop.

mriehema avatar Nov 05 '15 12:11 mriehema

Great information, thank you Michael. Maybe readFileAsync is not resolving? Can you tell if pkgData (promise) is resolving? Maybe add .catch to pkgData at the end of programRunLocal and see if an error is getting swallowed. Or try pkgData.then(console.log.bind(console, 'hi')) without the partial functional application, just to test a simpler case.

I wish I was able to reproduce and troubleshoot directly...

raineorshine avatar Nov 05 '15 18:11 raineorshine

@mriehema I added verbose logging to v2.40. Can you upgrade then report the output of the following command?

ncu --loglevel verbose

raineorshine avatar Nov 10 '15 15:11 raineorshine

Awesome! I would upgrade and test it, if I could. :(

$ npm install npm-check-updates -g

> [email protected] postinstall C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates\node_modules\snyk
> node cli/index.js help api-license

The use of Snyk's API, whether through the use of the 'snyk' npm package or otherwise, is subject to the terms & conditions specified here: https://snyk.io/policies/customer-agreement.pdf

C:\Users\mriehema\AppData\Roaming\npm\npm-check-updates -> C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates\bin\old-alias
C:\Users\mriehema\AppData\Roaming\npm\ncu -> C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates\bin\npm-check-updates

> [email protected] postinstall C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates
> npm run snyk-protect


> [email protected] snyk-protect C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates
> snyk protect

ENOENT: no such file or directory, open 'C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates\node_modules\bower\node_modules\handlebars\node_modules\uglify-js\.snyk-npm:uglify-js:20150824.flag'

npm ERR! Windows_NT 6.1.7601
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\mriehema\\AppData\\Roaming\\npm\\node_modules\\npm-check-updates\\node_modules\\npm\\bin\\npm-cli.js" "run" "snyk-protect"
npm ERR! node v4.2.2
npm ERR! npm  v3.3.12
npm ERR! code ELIFECYCLE
npm ERR! [email protected] snyk-protect: `snyk protect`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] snyk-protect script 'snyk protect'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the npm-check-updates package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     snyk protect
npm ERR! You can get their info via:
npm ERR!     npm owner ls npm-check-updates
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\mriehema\AppData\Roaming\npm\node_modules\npm-check-updates\npm-debug.log
npm ERR! Windows_NT 6.1.7601
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "npm-check-updates" "-g"
npm ERR! node v4.2.2
npm ERR! npm  v2.14.7
npm ERR! code ELIFECYCLE

npm ERR! [email protected] postinstall: `npm run snyk-protect`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] postinstall script 'npm run snyk-protect'.
npm ERR! This is most likely a problem with the npm-check-updates package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run snyk-protect
npm ERR! You can get their info via:
npm ERR!     npm owner ls npm-check-updates
npm ERR! There is likely additional logging output above.

mriehema avatar Nov 10 '15 16:11 mriehema

Snyk would be great if it worked out of the box, but it's not worth adding if it is going to break in some environments. I'll remove it.

raineorshine avatar Nov 10 '15 22:11 raineorshine

Try it now :). v2.4.1

raineorshine avatar Nov 10 '15 22:11 raineorshine

@metaraine shouldn't synk be a dev-dep and not linked to install? npm install shouldn't do anything with synk

ghost avatar Nov 11 '15 00:11 ghost

@metaraine Thanks for your support. So I checked it:

$ ncu --loglevel verbose
Initializing...
Running in local mode...
Finding local package file...
Waiting for package data on stdin...

BTW: I updated to node v4.2.2 and npm v2.14.7

So it looks like it still hangs up like I described. I added following code:

    pkgData.then(function(value) {
      console.log(value); // "Success!"
      throw 'oh, no!';
    }).catch(function(e) {
      console.log(e); // "oh, no!"
    }).then(function(e){
      console.log('after a catch the chain is restored');
    }, function () {
      console.log('Not fired due to the catch');
    });

Well... Same output as before. Just no response of the Promise.

mriehema avatar Nov 11 '15 09:11 mriehema

Success! After some more debugging I tried to run it with explicit packageFile, and it works!

$ ncu --loglevel verbose --packageFile package.json
Initializing...
Running in local mode...
Finding local package file...
Getting installed packages...
Fetching latest versions...

All dependencies match the latest package versions :)

Looks like pkgData = require('get-stdin-promise'); is not working as expected.

mriehema avatar Nov 11 '15 10:11 mriehema

The question is why didn't it find the local package.json when you run it without parameters. "Finding local package file..." was supposed to succeed so it shouldn't have tried to read it from stdin.

By the way, we can replace get-stdin-promise with get-stdin since they added builtin support for promises in v5.0.0.

etiktin avatar Nov 11 '15 15:11 etiktin

@etiktin npm-check-updates waits for stdin if process.stdin.isTTY is false. This is the proper way to check for stdin afaik, but please educate me if someone knows otherwise. npm-check-updates will look for a local package file only if it is not expecting anything from stdin. That message would probably be more accurate if it said "Finding package file data..."

@mriehema I wonder why isTTY is returning false when you are not piping anything to stdin. I googled "process.stdin.isTTY Windows" to see if it was something specific to your OS, but didn't turn up anything. Using --packageFile will work in the mean time. Thanks for your persistence!

@robertbaker snyk added itself to the package.json, so if anything is incorrect, it is due to snyk installing itself incorrectly, which for me is a good reason to hold off until they get that straightened out.

raineorshine avatar Nov 11 '15 17:11 raineorshine

FYI: Updated to ncu v2.5.1 but it still doesn't work without --packageFile

$ ncu -l silly
Initializing...
Running in local mode...
Finding package file data...
Waiting for package data on stdin...

mriehema avatar Nov 26 '15 14:11 mriehema

I'll keep this issue open since ncu still incorrectly waits for stdin in some environments. Adding an explicit --packageFile is a viable workaround.

As for the general performance issues, I fixed a bottleneck that should give a ~3x speed improvement (published in v2.5.3).

raineorshine avatar Dec 06 '15 18:12 raineorshine

Confirming hang on Windows 8 and Cygwin64.

Windows, being backwards, has no native concept of a TTY. I'm not going to dig into it myself as specifying the package file explicitly of course works, but unless any underlying package/implementation of isTTY is Cygwin-aware i wouldn't expect things to work out well.

However, it does work as expected under the native 'command processor' cmd.exe

mikemagnusw avatar Dec 07 '15 14:12 mikemagnusw

@mikemagnusw Thx for this infos. Updated to v2.5.4 and used ncu in Git Bash. Still same error. Used in cmd.exe. Works. Git Bash based on MinTTY. Found an issue according to STDIN, see https://github.com/mintty/mintty/issues/56

mriehema avatar Dec 08 '15 09:12 mriehema