apache
GH-2344: Yarn fails on windows CI-BUILD
GitHub issue resolved #2344
Pull request Description:
Looked at the links provided by @kinow in https://github.com/apache/jena/issues/2344#issuecomment-2021290428 and tried to change ' with \".
This gave a new error which looks like crossenv doesn't support := on windows, in ${FUSEKI_PORT:=3030} (and PORT).
Looked at crossenv on github and I think it is not available.
I looked at maybe changing the shell to run from in windows, by adding a .npmrc file, but then I got license (RAT) errors when running the CI, and instead of adding it to exclude list, I asked GPT figured the operator is probably not available in powershell. I ended up using the ENV vars only since FUSEKI_PORT is already used without fallback values in other locations of the package.json file.
Glanced at the final e2e output in https://github.com/OyvindLGjesdal/jena/actions/runs/8623049304 and it looks good to my untrained eyes.
Have left to check if it breaks anything for Linux and Mac, and will create a single commit.
Output when only fixing the apos -> escaped quote:
[INFO] $ cross-env FUSEKI_PORT="${FUSEKI_PORT:=3030}" PORT="${PORT:=8080}" concurrently --names 'SERVER,CLIENT,TESTS' --prefix-colors 'yellow,blue,green' --success 'first' --kill-others "yarn run serve:fuseki" "yarn wait-on http://localhost:${FUSEKI_PORT}/$/ping && yarn run dev" "yarn wait-on http-get://localhost:${PORT}/index.html && cypress run $@"
[INFO] [TESTS] $ C:\jena\jena-fuseki2\jena-fuseki-ui\node_modules\.bin\wait-on http-get://localhost:${PORT:=8080}/index.html
[INFO] [CLIENT] $ C:\jena\jena-fuseki2\jena-fuseki-ui\node_modules\.bin\wait-on http://localhost:${FUSEKI_PORT:=3030}/$/ping
[INFO] [SERVER] $ nodemon src/services/mock/json-server.js
[INFO] [SERVER] [nodemon] 3.1.0
[INFO] [SERVER] [nodemon] to restart at any time, enter `rs`
[INFO] [SERVER] [nodemon] watching path(s): *.*
[INFO] [SERVER] [nodemon] watching extensions: js,mjs,cjs,json
[INFO] [SERVER] [nodemon] starting `node src/services/mock/json-server.js`
[INFO] [SERVER] node:events:496
[INFO] [SERVER] throw er; // Unhandled 'error' event
[INFO] [SERVER] ^
[INFO] [SERVER]
[INFO] [SERVER] Error: listen EACCES: permission denied ${FUSEKI_PORT:=3030}
[INFO] [SERVER] at Server.setupListenHandle [as _listen2] (node:net:1855:21)
[INFO] [SERVER] at listenInCluster (node:net:1920:12)
[INFO] [SERVER] at Server.listen (node:net:2025:5)
[INFO] [SERVER] at Function.listen (C:\jena\jena-fuseki2\jena-fuseki-ui\node_modules\express\lib\application.js:635:24)
[INFO] [SERVER] at Object.<anonymous> (C:\jena\jena-fuseki2\jena-fuseki-ui\src\services\mock\json-server.js:294:8)
[INFO] [SERVER] at Module._compile (node:internal/modules/cjs/loader:1376:14)
[INFO] [SERVER] at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
[INFO] [SERVER] at Module.load (node:internal/modules/cjs/loader:1207:32)
[INFO] [SERVER] at Module._load (node:internal/modules/cjs/loader:1023:12)
[INFO] [SERVER] at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
[INFO] [SERVER] Emitted 'error' event on Server instance at:
[INFO] [SERVER] at emitErrorNT (node:net:1899:8)
[INFO] [SERVER] at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
[INFO] [SERVER] code: 'EACCES',
[INFO] [SERVER] errno: -4092,
[INFO] [SERVER] syscall: 'listen',
[INFO] [SERVER] address: '${FUSEKI_PORT:=3030}',
[INFO] [SERVER] port: -1
[INFO] [SERVER] }
[INFO] [SERVER]
[INFO] [SERVER] Node.js v20.11.0
[INFO] [SERVER] [nodemon] app crashed - waiting for file changes before starting...
- [ ] Tests are included.
- [ ] Documentation change and updates are provided for the Apache Jena website
- [ ] Commits have been squashed to remove intermediate development commit messages.
- [ ] Key commit messages start with the issue number (GH-xxxx)
By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the Contributor's Agreement.
See the Apache Jena "Contributing" guide.
Tested and ran on linux job which looks good, but confused me a bit first because the ordering of running the tests changed between runs. I guess the removal of the default operator may mean that the e2e becomes more clunky to run locally or standalone? FUSEKI_PORTS and PORTS changed between each run.
There are a few more errors after the tests for windows:
[INFO] --> Sending SIGTERM to other processes..
[INFO] [CLIENT] yarn wait-on http://localhost:%FUSEKI_PORT%/$/ping && yarn run dev exited with code 1
[INFO] --> Sending SIGTERM to other processes..
[INFO] [SERVER] yarn run serve:fuseki exited with code 1
[INFO] Done in 39.23s.
Adding a dependency for dotenv and using both dotenv and crossenv seems to work for the CI-Job, and the variables should be used by default from dotenv when working with it directly.
I don't understand why I have to use cross-env, if I already have the env set, but it feels like the values aren't available if not listed first in the script section, prior to the job. I am clearly missing something here in how this works.
Good news is that the windows job is passing and running tests even when turning on errors again in maven. Bad news is the script is kind of ugly and hackish, and adds an extra dependency (BSD). Will clean up the extra files and unused parts and push to a single commit again hopefully this evening.
Node added a parameter for reading environment vars without using a library in version 20.6, but I don't think it is reachable when using yarn, from web searches.
Another alternative that looks better imo, is writing a simple helper script that sets environment vars (with defaults) and adding it at the beginning. See https://stackoverflow.com/questions/32446734/how-to-set-environment-variables-in-a-cross-platform-way
That could mean removing a dependency (cross-env) instead of adding a dependency, however I didn't get it to work.
Added some comments https://github.com/apache/jena/issues/2344#issuecomment-2132421238
Here's main on Linux:
- - - │
[TESTS] └────────────────────────────────────────────────────────────────────────────────────────────────┘
[TESTS] ✔ All specs passed! 00:19 13 13 - - -
[TESTS]
[TESTS] yarn wait-on http-get://localhost:${PORT}/index.html && cypress run $@ exited with code 0
--> Sending SIGTERM to other processes..
[SERVER] yarn run serve:fuseki exited with code SIGTERM
--> Sending SIGTERM to other processes..
[CLIENT] yarn wait-on http://localhost:${FUSEKI_PORT}/$/ping && yarn run dev exited with code SIGTERM
Done in 35.64s.
Is the PR up to date before running? From line 3 in the screenshot, the script should also include the fallback values for the port for nix, which is not present in the image.
cross-env FUSEKI_PORT=\"${FUSEKI_PORT:=3030}\" PORT=\"${PORT:=8080}\"
for some reason the :=3030 and :=8080 part isn't showing.
The logic is that the windows and linux jobs should stay almost the same as before, while there is a different logic for windows.
Seems to work for me - this PR, rebased to current main.
yarn run test:e2e runs in 20.74s.
I've also run the PR/rebased with github actions. It got passed jena-fuseki-ui on Windows
The Windows GH action is unreliable at best especially at busy times (weekday day-evening). This is not new.
Looking good to me.
