Add support for Windows NPM

[MetaMmodern]

Description

These changes add support for Windows OS. Was tested on Windows 10 (x64), server and web-client start now.

:warning: I am not a haskell developer!: My code may be ugly and even disguisting, but that's what I've achived by googling)

Code explanation

1. The reason why npm was not starting is because Windows Process API appends .exe automatically to the executable filename, however npm as program does not provide any .exe file, only .cmd and .ps1. (explained here: https://stackoverflow.com/questions/43139364/createprocess-weird-behavior-with-files-without-extension) The first thing to do is to support .cmd for windows only. That's implemented in oSSpecificNpm variable. (Screenshot of ghci session for proof)

2. When we run npm.cmd from inside haskell it will complain that it can't find the npm-cli. The reason and solution is explained here: https://stackoverflow.com/a/44820337 . The implementation is made in compileOsSpecificNodeCommand Common function and in src/Wasp/Generator/Job/Process.hs file. (Screenshot as demo)

Things to do further

Do NOT update docs and do not mention Windows support, since there are still following issues persistent(at least in my env):

1. Some path's in generated js files are incorrect for windows (backslashes do not work).

2. Some issues with db and/or with encoding during db initialization/migration.

3. UI did not render for me😢.

Fixes #48

Type of change

Please select the option(s) that is more relevant.

• [x] Bug fix (non-breaking change which fixes an issue)
• [x] This change requires a documentation update

[Martinsos]

Ah I realize now you explained in the description why we are doing the cmd.exe thing! Ok I will check it out sorry about that -> I have to run now but I should be able to get back to it in in an hour or two, I will read it then and write more educated comment!

[Martinsos]

Alright, read about cmd.exe and left the comment, that is it from me for this first iteration of review!

[Martinsos]

Oh btw we use ormolu for formatting Haskell code -> is that something you could easily set up, so that we can get the CI working? If not, don't worry, I can take care of that for you later.

[MetaMmodern]

Oh btw we use ormolu for formatting Haskell code -> is that something you could easily set up, so that we can get the CI working? If not, don't worry, I can take care of that for you later.

Um, I'll try but not now, later evening)

[Martinsos]

In the meantime I made commits that should fix the two problems that you identified @MetaMmodern (wrong path and problem with \xad) -> I pushed them onto your main branch, so make sure to pull it before you continue. This should enable us to move forward again :)! I believe there will be another problem, which is the fact that script command does not exist on Windows, and we currently need it for smth, but let's see if that is a problem or not first and then we can consider it more.

[MetaMmodern]

@Martinsos for db migration similar error:

So wasp start starts, but because db is empty it's stuck at Loading....

For the rest review:

1. Namings changed.
2. Comments left.
3. Files Linted.

(cabal dependencies installation is extremely slow, even npm is not so slow. shame on cabal)

[Martinsos]

Awesome @MetaMmodern !

I left couple more comments, some are just comments/naming, one is about where to call the "buildNpmCmdWithArgs" function.

As for encoding issue with prisma db migrate -> that happens now due to us not supporting the encoding that you use in your terminal, which is CP866 (russian cyrilic). I found this library now https://hackage.haskell.org/package/encoding-0.8.7/docs/Data-Encoding.html, once we add that we should have support for CP866 and many other encodings -> I just need to find some time to do it properly since it is not super simple due to them complicating things a bit with Monads!

Ha yes it takes long time for cabal since there is a lot to compile really, Haskell is more complex language than JS, but usually it is so slow only on the first build!

[MetaMmodern]

@MetaMmodern this should be it! In order to speed the merging up a bit, I fixed one tiny compiler error that somehow sneaked in, did a tiny bit of left-over refactoring, and I added more encodings, that should resolve the issue you had with CP861!

Would you like to give it a finally try, to see if the issue with CP861 is now resolved?

in an hour or so) lil bit later

[MetaMmodern]

@Martinsos what's with build?

[Martinsos]

@MetaMmodern hm good question, it doesn't seem to show any error messages -> I restarted it, let's see if that helps!

[MetaMmodern]

@Martinsos still the same. Check raw logs maybe. It also skips code formatting step, idk if it's intentional.

[Martinsos]

@MetaMmodern I think I found the reason -> there was this flag for this new encoding package that sometimes causes problems. I turned it off since we don't use it -> let's see now if it helps. What sucks is that encodings package takes really long time to build, 5-10 minutes, I am not sure why, it is reported on their GH repo as an issue but not resolved yet. But there aren't really any better alternatives out there.

TODO for myself: maybe try with the knob package -> wrap bytestrings into file handlers and let withUtf8 handle it. But does withUtf8 support wide array of encodings or not?

[MetaMmodern]

@Martinsos maybe there is a way to apply caching to library builds? Like separate those into some kind of .bin files and use cache if dependencies did not change)

[Martinsos]

@Martinsos maybe there is a way to apply caching to library builds? Like separate those into some kind of .bin files and use cache if dependencies did not change)

Yup there is caching already in place! It just takes long this first time.

[Martinsos]

It builds now @MetaMmodern

[MetaMmodern]

It builds now @MetaMmodern

ok. checking locally

[MetaMmodern]

@Martinsos nnnnnope)

[Martinsos]

Wooo great! This was unfortunately expected, but we took care of encoding yay! This is furthest we got right?

On Mon, 4 Jul 2022, 18:06 Bunyamin Shabanov, @.***> wrote:

@Martinsos https://github.com/Martinsos nnnnnope) [image: image] https://user-images.githubusercontent.com/42899786/177189747-06a070ad-33b4-41e1-90c7-192101fb550b.png

— Reply to this email directly, view it on GitHub https://github.com/wasp-lang/wasp/pull/658#issuecomment-1173969937, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALXFB32RHA74DFD325MC33VSMDWXANCNFSM52GXRSUQ . You are receiving this because you were mentioned.Message ID: @.***>

[MetaMmodern]

Wooo great! This was unfortunately expected, but we took care of encoding yay! This is furthest we got right?

On Mon, 4 Jul 2022, 18:06 Bunyamin Shabanov, @.***> wrote:

@Martinsos https://github.com/Martinsos nnnnnope) [image: image] https://user-images.githubusercontent.com/42899786/177189747-06a070ad-33b4-41e1-90c7-192101fb550b.png

— Reply to this email directly, view it on GitHub https://github.com/wasp-lang/wasp/pull/658#issuecomment-1173969937, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALXFB32RHA74DFD325MC33VSMDWXANCNFSM52GXRSUQ . You are receiving this because you were mentioned.Message ID: @.***>

Well, as I understood--yes.

[Martinsos]

I managed to set up my Windows to run Haskell, so now I can also test!

Ok, so I did following, as a quick hack in the place where we now use scriptand it seems to be enough to get prisma db migrate-dev to work (at least for now):

let job = runNodeDependentCommandAsJob J.Db serverDir ("cmd.exe", pure $ unwords $ "/c" : npxPrismaMigrateCmd)

I will try to properly add this to the code.

Regarding your encoding @MetaMmodern -> is there a way I can also set my encoding to be like yours, so I can test? I tried setting language to Croatian, and to Russian, but it uses CP437 in that case and I don't have any issues with encodings, even if I just do decodeUTF8 -> ok, escape characters are often not correctly shown, but it doesn't break. So how can I reproduce it to have the same situation as you? Because I am not happy with the current encoding solution I made, it works but it has its problems.

[MetaMmodern]

@Martinsos hey, congrats on setting up Windows. For encoding: try running chcp 866 in cmd window. Should work. Got it from here: https://superuser.com/questions/1170656/windows-10-terminal-encoding

[Martinsos]

Thanks, I tried that @MetaMmodern ! But I can't reproduce the failure to parse, the one with \xad error.

I did a final commit, which:

1. Removes usage of script for Windows -> for me wasp db migrate-dev now works! But it didn't ask me any questions. I know sometimes it asks, and I wonder if in that case it will not work. But I don't know how to get it to ask. Maybe you can try running wasp db migrate-dev, see how it goes?
2. I reversed encoding solution to the most simple one, which works for me regardless of encoding. I wonder if this will still work for you, or will cause problems again.

TLDR; Can you give it one last spin? I did as much as I could on my Windows, for me it works now, but I wonder if it will work for you with your CP861 encoding (worked for me when I used chcp 861 but it might be different).

TODO for me: I have to decide what to do with encoding code. Probably kick out encoding library, and go with something simple, either just decodeUtf8Lenient, or use knob.

[MetaMmodern]

@Martinsos What does this mean? Do I need to have Postgres Installed to run WASP? Do I need to manually add this env variable or is it set by WASP CLI?

[Martinsos]

Oho awesome you got this far!!! So it all works!

Yes it seems you need postgre database and provide its connection URL via this variable. Which app are you running? In docs you have detailed instructions on how to run postgre dB, includin one liner to get it going quickly via docker.

On Tue, 5 Jul 2022, 14:27 Bunyamin Shabanov, @.***> wrote:

@Martinsos https://github.com/Martinsos What does this mean? [image: image] https://user-images.githubusercontent.com/42899786/177327181-05b5ff18-a0b1-429f-96e1-8e4cd699b059.png Do I need to have Postgres Installed to run WASP? Do I need to manually add this env variable or is it set by WASP CLI?

— Reply to this email directly, view it on GitHub https://github.com/wasp-lang/wasp/pull/658#issuecomment-1175001065, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALXFB22MAPAXESN5ENKIPLVSQS3HANCNFSM52GXRSUQ . You are receiving this because you were mentioned.Message ID: @.***>

[MetaMmodern]

@Martinsos okay, I'll try it now.

[MetaMmodern]

@Martinsos In Ukraine we call this "PEREMOGA", which means "a win"💪🏼 Migration done.

App running--done.

Important note for you to put into documentation: If you run docker on Windows using WSL you have to specify WSL host, not localhost as your host for DB connection. You can get your host using this command: wsl hostname -I. Source is here: https://jkarelins.medium.com/run-postgresql-database-in-docker-container-on-wsl2-656ed3c02280

So that's it)

[Martinsos]

Awesome @MetaMmodern well peremoga it is then :D, I can't believe we have Windows support working!! Thanks again, without you pushing this, we would not have it working soon for sure. Let me keep this PR open few more days so I can do some final tweaks regarding the encodings + rebase it on newest master since some changes are coming there that we will need to adapt to, but everything important is done! Yay :)

[Martinsos]

@MetaMmodern , when you catch a moment (no hurry), could you run one last test? I now made a change in the code we use for decoding, so it is less convoluted + doesn't need external library -> in theory it should be the same, but if you can give it a try (wasp db migrate-dev) I would feel sure it works. Unfortunately I still haven't managed to replicate your situation with encoding in order to test the situation you had.

Btw is your output messy in any way? For me, on Windows, it is often not colored when it should be, plus some chars get replaced with ?, the exotic ones.

[MetaMmodern]

@Martinsos anything for you my man)))

Yes, I have artifacts on output. Yes, I have question marks (?) at the beginning of the lines, just check my previous screenshots. But colouring works.

[MetaMmodern]

@Martinsos looks fine to me. well, except the question marks.