Windows 64bit error on build and serve

[mjosborne1]

Downloaded the Windows 64 bit version plenti_0.1.26_Windows_64-bit.tar.gz

[Created a site using plenti on windows:]

PS C:\Code\www\examples> plenti new site plenti-test-site Created plenti site scaffolding in "plenti-test-site" folder Installing NPM dependencies... npm notice created a lockfile as package-lock.json. You should commit this file. added 4 packages from 3 contributors and audited 5 packages in 2.639s found 0 vulnerabilities

NPM install complete! PS C:\Code\www\examples> cd .\plenti-test-site\

When i run either serve or build I get the same error...

PS C:\Code\www\examples\plenti-test-site> plenti serve Total build took 20.0303ms panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]: plenti/cmd/build.DataSource.func1(0xc00001c940, 0x1c, 0xa54c00, 0xc000182700, 0x0, 0x0, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/build/data_source.go:44 +0x151a path/filepath.walk(0xc00001c940, 0x1c, 0xa54c00, 0xc000182700, 0xc00010fa00, 0x0, 0x0) /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:360 +0x42c path/filepath.walk(0xc000020b20, 0xc, 0xa54c00, 0xc0001824d0, 0xc00010fa00, 0x0, 0x0) /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:384 +0x306 path/filepath.walk(0x985c03, 0x7, 0xa54c00, 0xc000182310, 0xc00010fa00, 0x0, 0x0) /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:384 +0x306 path/filepath.Walk(0x985c03, 0x7, 0xc00010fa00, 0xf, 0xf) /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:406 +0x106 plenti/cmd/build.DataSource(0xc000086480, 0x6, 0xc000086460, 0x6, 0xbb8, 0xc00009c4e0, 0x0, 0x0, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/build/data_source.go:40 +0x3dd plenti/cmd.Build() /home/runner/work/plenti/plenti/cmd/build.go:92 +0x3ad plenti/cmd.glob..func1(0xda7e60, 0xdddf08, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/build.go:43 +0x27 plenti/cmd.glob..func4(0xda7e60, 0xdddf08, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/serve.go:55 +0x54f github.com/spf13/cobra.(*Command).execute(0xda7e60, 0xdddf08, 0x0, 0x0, 0xda7e60, 0xdddf08) /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:842 +0x2a4 github.com/spf13/cobra.(*Command).ExecuteC(0xda7bc0, 0x444e01, 0xd6a100, 0xc000043f78) /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:943 +0x31e github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:883 plenti/cmd.Execute() /home/runner/work/plenti/plenti/cmd/root.go:33 +0x38 main.main() /home/runner/work/plenti/plenti/main.go:8 +0x27

[jimafisk]

Thanks for the bug report, super helpful to have someone testing this out on Windows! I'm not sure what's happening here off the top of my head, so I'll have to investigate. I had only tested with Mac and Linux so I'm not entirely surprised. It's weird that we're getting a goroutine panic, I would have expected a nodejs error when compiling the svelte components.

I'm actively trying to finish removing the node dependency which rewrites a lot of the build, so I'd be curious if that release solves this issue before digging in too deep. Will loop back on this soon. Thanks!

[mjosborne1]

No worries, I'm updating my golang version to 1.14.6 to see if that helps.

[mjosborne1]

I tried this another way using docker...

C:\Code\www\examples\plenti-test-site>docker run plentico/plenti:latest serve Unable to read site config file: unexpected end of JSON input Unable to read site config file: unexpected end of JSON input Total build took 788.3µs panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x5480a8]

goroutine 1 [running]: plenti/cmd/build.AssetsCopy.func1(0x983628, 0x6, 0x0, 0x0, 0xa489e0, 0xc00013c060, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/build/assets_copy.go:23 +0xc8 path/filepath.Walk(0x983628, 0x6, 0xc000127a50, 0x0, 0x0) /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:404 +0x6a plenti/cmd/build.AssetsCopy(0xc0000a3160, 0x6) /home/runner/work/plenti/plenti/cmd/build/assets_copy.go:21 +0x160 plenti/cmd.Build() /home/runner/work/plenti/plenti/cmd/build.go:89 +0x34e plenti/cmd.glob..func1(0xdacea0, 0xde11a0, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/build.go:43 +0x20 plenti/cmd.glob..func4(0xdacea0, 0xde11a0, 0x0, 0x0) /home/runner/work/plenti/plenti/cmd/serve.go:55 +0x548 github.com/spf13/cobra.(*Command).execute(0xdacea0, 0xde11a0, 0x0, 0x0, 0xdacea0, 0xde11a0) /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:842 +0x29d github.com/spf13/cobra.(*Command).ExecuteC(0xdacc00, 0x44228a, 0xd6eee0, 0xc000036778) /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:943 +0x317 github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:883 plenti/cmd.Execute() /home/runner/work/plenti/plenti/cmd/root.go:33 +0x31 main.main() /home/runner/work/plenti/plenti/main.go:8 +0x20

Is there a missing config file?

e.g. Where would this live on windows.... Global Flags: --config string config file (default is $HOME/.plenti.yaml)

[jimafisk]

I just installed plenti using updated instructions for scoop: https://github.com/plentico/plenti/issues/44#issuecomment-660400346. I also added Node v14.5.0 via scoop install nodejs. I'm getting the same error as you.

Total build took 69.5391ms
panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
plenti/cmd/build.DataSource.func1(0xc00001f0c0, 0x1c, 0xa54c00, 0xc000183030, 0x0, 0x0, 0x0, 0x0)
        /home/runner/work/plenti/plenti/cmd/build/data_source.go:44 +0x151a
path/filepath.walk(0xc00001f0c0, 0x1c, 0xa54c00, 0xc000183030, 0xc00010fa00, 0x0, 0x0)
        /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:360 +0x42c
path/filepath.walk(0xc000019330, 0xc, 0xa54c00, 0xc000182e00, 0xc00010fa00, 0x0, 0x0)
        /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:384 +0x306
path/filepath.walk(0x985c03, 0x7, 0xa54c00, 0xc000182c40, 0xc00010fa00, 0x0, 0x0)
        /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:384 +0x306
path/filepath.Walk(0x985c03, 0x7, 0xc00010fa00, 0xf, 0xf)
        /opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:406 +0x106
plenti/cmd/build.DataSource(0xc000018f10, 0x6, 0xc000018ef0, 0x6, 0xbb8, 0xc00018a5d0, 0x0, 0x0, 0x0, 0x0)
        /home/runner/work/plenti/plenti/cmd/build/data_source.go:40 +0x3dd
plenti/cmd.Build()
        /home/runner/work/plenti/plenti/cmd/build.go:92 +0x3ad
plenti/cmd.glob..func1(0xda7e60, 0xdddf08, 0x0, 0x0)
        /home/runner/work/plenti/plenti/cmd/build.go:43 +0x27
plenti/cmd.glob..func4(0xda7e60, 0xdddf08, 0x0, 0x0)
        /home/runner/work/plenti/plenti/cmd/serve.go:55 +0x54f
github.com/spf13/cobra.(*Command).execute(0xda7e60, 0xdddf08, 0x0, 0x0, 0xda7e60, 0xdddf08)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:842 +0x2a4
github.com/spf13/cobra.(*Command).ExecuteC(0xda7bc0, 0x444e01, 0xd6a100, 0xc000043f78)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:943 +0x31e
github.com/spf13/cobra.(*Command).Execute(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:883
plenti/cmd.Execute()
        /home/runner/work/plenti/plenti/cmd/root.go:33 +0x38
main.main()
        /home/runner/work/plenti/plenti/main.go:8 +0x27

It even spit out a mess of other things possibly related to trying to watch the files for changes.

You shouldn't need Go installed to run plenti, but I'm on v1.14 too if you're interested in working on the project :).

[mjosborne1]

I think it has the right concept, and this kind of thing is what I'm looking at using particularly to change the website on the fly. So if I have some spare time I'll try and help out.

[jimafisk]

Awesome! If there's anything I can do to help get you set up just let me know. I'll spend some more time with Window testing after I've got the rewrite of the build out the door.

[mjosborne1]

I've debugged plenti on Windows in VS code and the go server is defintiely expecting a site config file somewhere...

API server listening at: 127.0.0.1:46949 Unable to read site config file: unexpected end of JSON input Total build took 6.0068ms Process exiting with code: 0

[jimafisk]

Ah yes, good sleuthing! There should be a site configuration file in the root of the project named "plenti.json" that by default looks like this: https://github.com/plentico/plenti/blob/master/defaults/plenti.json. It's possible that it exists in your new project, but plenti is not doing a good job of figuring out the path on Windows to find it.

It looks like we're getting an error "unmarshalling" the JSON from this file (putting the JSON info into a Go struct that we can use): https://github.com/plentico/plenti/blob/master/readers/site_config.go#L25. I should add error handling on the line above where we're actually reading the file in to see if there are any issues there first.

[jimafisk]

Plenti v0.2.0 and newer versions use v8go to run the build without requiring NodeJS on your computer. Unfortunately Windows is not currently supported by that package, you can track progress on that here. There might be additional issues in using plenti on windows, but we can't even build Windows binaries until this is resolved. If folks want Windows support, go show your love and support for that project, or even think about contributing over there!

[rdricco]

I wanna try Plenti but I'm on a windows machine. What's the best install option?

[jimafisk]

Hey @rdricco, thanks for taking a look! Unfortunately the Windows version has a couple of blocking issues that prevent it from working at the moment. This is something I'd like to resolve in the near future, but we need the issue with v8go mentioned above resolved first. It's possible in the meantime I could get this working using the legacy NodeJS build: plenti serve --nodejs=true but I'll have to resolve some bugs there as well.

[GerwarKi]

An alternative to fix it is to use plenti under a WSL VSCode helps a lot with the "Remote - WSL" extension. Now I am with SUSE, it can be installed from the windows store. @jimafisk, for now while the bug is not solved you can add a quick guide on the plenti page of how to do this so that those who develop in windows don't arrive here.

[rdricco]

Nice! WSL v1 @GerwarKi ? I'll give a try...

[jimafisk]

I didn't know about WSL, very cool! I agree the docs should be better so folks don't have to hunt for this, have you had a chance to install plenti this way @GerwarKi? If you have steps that worked for you I'd be happy to add them the to README and the website docs. Otherwise I'll try to play around with this soon and see if I can get it working. Thanks!

[awulkan]

It seems like the v8go project has been abandoned for 9 months now, and no solution for Windows support in sight.

The Plenti website still links to a Windows file on the download page, but there is no Windows file anymore. Maybe it's best to just remove the Windows download option on the website so that people like me don't have to search through Github issues to find answers.

[Fer17]

After testing your amazing product in Linux (with great success) I've tried to run it in our Windows Server,.. without success. I've found this issue about Windows is still open. What a pity it doesn't work fine there. Is there any solution on the roadmap, dear Jim? Perhaps replacing the v8go by NodeJs instead? Thank you in advance.

[jimafisk]

@awulkan sorry the download option on the website was confusing, Windows used to work with earlier versions of Plenti that used NodeJS for builds. I just updated the website so hopefully the current lack of support is clear. I looked into several other options when evaluating v8go, and it seemed to be the fastest option by a significant margin.

@Fer17 I wish I had a better short term plan for Windows support. The best advice I can offer at the moment is to show some support to the v8go project by starring their repo and giving the maintainer some encouragement. If anyone has Windows + Go experience, helping out over there with the Windows v8 build would resolve this: https://github.com/rogchap/v8go/issues/7.

We do still have an option (for now) to run NodeJS builds: plenti build --nodejs=true. The problem is I had stopped building Windows binaries entirely, so while nodejs builds should work on Mac and Linux, they won't on Windows at the moment. I'm not sure if I can cross compile to Windows at all given the outstanding v8go issue, but I'll look into this to see if it can be used as a short term solution.

Thank you both for the input! I really hope to get a Windows version out soon, I think our best path is helping get Windows support on the v8go project.

[leaanthony]

My recommendation is to release a lightweight docker image with the tools preinstalled. On windows, pull this image and use it for the build by default. It will save you a lot of headaches, trust me!

[awulkan]

It's not really a problem for Windows 10 at least. It has WSL2, so we can just use the Linux version. But it would be nice if it was documented.

[G2G2G2G]

nv GOOS=windows GOARCH=amd64 go build .
# plenti/cmd/build
cmd/build/client.go:18:13: undefined: v8go.Context
cmd/build/client.go:42:12: undefined: v8go.NewContext
cmd/build/client.go:48:14: undefined: v8go.NewContext
cmd/build/client.go:123:25: undefined: v8go.Context

can't compile it on linux for windows either, and I think your issue might be in part of https://github.com/rogchap/v8go/issues/7 not actually plenti? I always have issues with C++/C -> go and expecting it to be cross platform, I don't think I've seen anything that works. It only works on like 1 or 2 systems.

oh wait I can't read you already linked that issue above, as you mentioned in the last post of that thread, the build being dependent that's true. It is in fact a very big issue in C cross compiling. Which is why Java, Go and C# are so famous/amazing/popular... If C didn't have these issues I do not think the above 3 languages would be nearly as popular.

Also WSL is a great option and has a better dev environment on windows than native windows does (for most things, not all). So that's a real option if people want to use it, some may call it inconvenient though since they're used to the microsoft inconvenient by design experience™

[dp-op]

Thanks @jimafisk for this well thought out product. I was able to successfully install it on Ubuntu 20.04 on my window machine via WSL2 (windows subsystem for linux). David Bombal explains it in his how-to video Nevertheless, I will definitely pursue plenti further with the great hope of a working version directly for windows.

[jimafisk]

Thanks @dp-op! I'm new to the whole WSL concept, but that video really helped clear things up and I'm totally on board for Linux kernel in windows! For someone like me who is more comfortable with unix style commands, I'd honestly prefer to use Plenti this way than using it natively in the windows command line or powershell.

There is good news for folks who want to use Plenti directly with Windows though! Looks like a PR to add windows support was just opened on the v8go project: https://github.com/rogchap/v8go/pull/46. If anyone reading this wants this feature in Plenti, go give that PR a review and a thumbs up! Thanks!

[rogchap]

I've just released v8go v0.3.0 which provides support for Windows; but unfortunately it's via a dynamic link to v8 rather than the static link for Darwin and Linux. Might mean a few extra steps for getting this working on Windows.

[jimafisk]

That's awesome @rogchap, congrats! I know a lot of folks on this project will be excited to see Windows support so I'll take a look.

[neptoess]

Since plenti is doing binary distribution, the easiest thing to do is likely:

1. Build using the MSYS2 Mingw-w64 toolchain
2. Using ntldd (provided by the mingw-w64-x86_64-ntldd-git pacman package in case it's not installed for whatever reason) to see the dynamic linking dependencies, copy the DLLs that aren't bundled by Windows, i.e. the ones that are in C:\Windows\System32, along with snapshot_blob.bin into the same folder as plenti.exe

Shipping DLLs this way has been pretty standard fare for Windows for decades now.

[jimafisk]

Hi @neptoess, thanks for making this PR! Are those setup instructions for someone building a binary from a Windows host machine? Ideally we'd have this pulled into our Goreleaser + GitHub Actions workflow where we're currently building Linux and Mac binaries from a virtual Linux machine. With this update are we able to extend our CI so the Linux host also builds a Windows binary?

[neptoess]

Those instructions are for building on Windows. Cross-compiling from Linux using MinGW is possible (@ed-alertedh does this with https://github.com/edenhill/librdkafka if I understand correctly), but you will need to build v8 that way as well, since my PR is relying on an MSYS2 pacman package. Windows build hosts with MSYS2 work great on Travis CI. I see Goreleaser supports Windows builds as well. I would suggest giving that a shot and falling back to the Linux cross-compiling only if it looks like it will be easier/cleaner.

[jimafisk]

Maybe Zig could help with CGO cross-compiling: https://dev.to/kristoff/zig-makes-go-cross-compilation-just-work-29ho

[kpym]

@jimafisk Probably a silly question, but why not use goja, which is written in pure go, instead of v8? Is it just for performance reasons, or are there some incompatibilities?

[jimafisk]

Not a silly question, I've definitely had my eye on Goja for a while. You're exactly right though, running the whole compile step in Goja is quite slow compared to V8. Here are some benchmarks: https://github.com/dop251/goja/issues/2#issuecomment-426429140

From Goja's README:

If most of the work is done in javascript (for example crypto or any other heavy calculations) you are definitely better off with V8. If you need a scripting language that drives an engine written in Go so that you need to make frequent calls between Go and javascript passing complex data structures then the cgo overhead may outweigh the benefits of having a faster javascript engine.

Running the Svelte compiler in its entirety is putting us in the first scenario at the moment. However, the interesting thing about the Svelte compiler is that although it's written in JS, it doesn't actually have to execute JS code besides the render to HTML step. So theoretically you could offload the lexer/parser work to Go and just use Goja to render the JS bits for a very fast compiling experience - but we'd be building our own Svelte at that point. If we can get enough interest/traction we'd consider building that (it's a large undertaking), but for now there are several low hanging fruit ways we're planning to improve the compile step, unfortunately they involve staying with V8 in the short term.