Get `npm install` during `wasp start` to output more details

[Martinsos]

This issue is coming from #207 -> take a look there for more details.

Summary is: when you run wasp start, you get stdout and stderr from web app and server. However, while npm install is happening for both of these, very little output is coming through -> only warnings. There is no information about which packages are being installed and how is that progressing. This is bad because npm install can take half a minute, minute, even more, and there is no output, so it feels like something is stuck.

On the other hand, if you run npm install manually, you will see a lot of output happening. So, I am assuming that npm install detects it is being run by another program (not interactively) and therefore reduces amount of output. We should find a way to make it output more information!

Ideas:

• maybe there is a flag that can be passed to npm install to have it output more info?
• we can use script to cheat npm install into thinking it is being run interactively, although that might result with messy output potentially (due to us having multiple sources of output). We are already doing this with prisma commands so if we are going with this we should re-use that code.

First option would be ideal, but if it doesn't work we use the second one.

[faassen]

I wonder whether it's possible to use npm as a library. This seems to exist: https://github.com/npm/libnpm#table-of-contents

Of course that won't help us immediately in Haskell.

[Martinsos]

We tried doing this in the following PR https://github.com/wasp-lang/wasp/pull/260 but haven't solved it yet - it is still open!

[Martinsos]

With npm 8, there is --loglevel flag.

So while we still haven't figured out how to get npm install to behave in interactive manner, at least this way we can actually get more output.

Default is --loglevel notice, but that doesn't print almost any messages, so that is not ao solution. Next level, http, on the other hand, prints a ton of messages. Since next three are http, timing, and info, and ton of messages is get printed even with http, I decided to go with info -> yeah, it prints a lot of messages, but progress is visible, and at least it gives detailed info on what is happening.

This was done in commit ccf3c7199f2674ede72324ac99e59c94f97f5fab .

I think solves this to some degree for now.

I won't close the issue though, as I think we could still find a nicer way for this. Ideally, we would get less logs, or we would get the same output as when npm is interactive, or smth else (some progress bar or smth).

[Martinsos]

At the end, we reverted the loglevel solution above -> it was just too much output. Instead we introduced nice progress indicator (messages about how install is still progressing).

I believe this is good enough solution for now! It shows that install is happening + only important / relevant logs are forwarded, which is good.

[sodic]

I'm reopening this since we're slowly outliving the original short-term solution.

[Martinsos]

I'm reopening this since we're slowly outliving the original short-term solution.

I believe you have good reasons, but can you list them there, how is current short-term solution failing currently?

[sodic]

Sure, it's what we talked about in the presentation a couple of weeks ago, but it's mostly the same reasons we had the first time around (plus a few extra):

Comparisons with Next and T3 fall short (and we also had users complain) when it comes to the following:

• a lot of vertical space taken
• no indication of real progress
• no indication when the internet isn't working

[Martinsos]

Sure, it's what we talked about in the presentation a couple of weeks ago, but it's mostly the same reasons we had the first time around (plus a few extra):

• user complaints
• comparisons with Next and T3 fall short
• a lot of vertical space taken
• no indication of real progress
• no indication when the internet isn't working

Can you maybe reference the complaints here? Links to them, or directly copy them, or paraphrasing?

What does it mean it falls short in comparison with Next?

[Martinsos]

Usually when people have slow internet connection, npm install will take quite long, and they will just be seeing the same message from us about "waiting a bit longer". After seeing such message 10 times, people start feeling stuck! And we are not showing what is happening in the background.

What we should do is somehow show what is happening in the background. I am not sure how, as we are not getting any more output from npm install to show. Maybe we could increase its verbosity level, would that give us more messages? And we can maybe even filter those verbose ones from non-verbose ones and start showing verbose ones only if npm install is taking long?