Windows binary MODULE_NOT_FOUND

[maxb2]

trafficstars

See dungeon-revealer/dungeon-revealer#1545.

Windows is periodically deleting individual files from the temporary directory that the caxa binary is extracted into. This means that the extracted node binary tries to run in a broken project directory which throws MODULE_NOT_FOUND errors. The current workaround for our project is to delete the caxa temporary directory and rerun the initial extraction. This only works for us because we don't store the user data in the caxa temporary directory.

Example error:

D:\DnD\Dungeon Revealer>"D:\DnD\Dungeon Revealer\dungeon-revealer-windows.exe"
node:internal/modules/cjs/loader:930
  throw err;
  ^

Error: Cannot find module 'C:\Users\SCrid\AppData\Local\Temp\caxa\applications\dungeon-revealer-windows\3bn3q6shhl\0\server-build\index.js'
←[90m    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:927:15)←[39m
←[90m    at Function.Module._load (node:internal/modules/cjs/loader:772:27)←[39m
←[90m    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12)←[39m
←[90m    at node:internal/main/run_main_module:17:47←[39m {
  code: ←[32m'MODULE_NOT_FOUND'←[39m,
  requireStack: []
}

[n1ru4l]

This might be related to this: https://winaero.com/clear-temporary-files-automatically-windows-10/

A possible workaround would be to extract the files relative to the binary or to safe locations such as the user directory 🤔

[leafac]

Hi @maxb2,

Great to have another detailed report from you.

This is a rather annoying issue. It seems like temporary directories can’t be trusted.

I don’t think we have any control over the settings mentioned by @n1ru4l, so I think that the solution may be https://github.com/leafac/caxa/issues/20. This is unfortunate, because then the user is responsible for cleaning the caxa directory as stuff accumulates in there.

This only works for us because we don't store the user data in the caxa temporary directory.

Storing application data on the caxa temporary directory would be discouraged in any platform. Even if the operating system isn’t cleaning the temporary directory during execution, it will on reboot.

[saostad]

I have the same problem with windows 10. Also don't think this issue is related to the other issue you mentioned earlier, I checked the temp folder all the files exist. OS didn't clean anything!

internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module 'C:\Users\SOstad\AppData\Local\Temp\caxa\applications\x\gsqfezubgv\0\dist-esm\helpers\util' imported from C:\Users\SOstad\AppData\Local\Temp\caxa\applications\x\gsqfezubgv\0\dist-esm\index.js
    at new NodeError (internal/errors.js:322:7)
    at finalizeResolution (internal/modules/esm/resolve.js:308:11)
    at moduleResolve (internal/modules/esm/resolve.js:731:10)
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:842:11)
    at Loader.resolve (internal/modules/esm/loader.js:89:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:242:28)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:76:40)
    at link (internal/modules/esm/module_job.js:75:36) {
  code: 'ERR_MODULE_NOT_FOUND'

[leafac]

@saostad That’s spooky. Does the application start working and then fails with a missing path? If Windows didn’t clean files in the temporary directory, and the files are still there, then what do you think the error could be?

[saostad]

@saostad That’s spooky. Does the application start working and then fails with a missing path? If Windows didn’t clean files in the temporary directory, and the files are still there, then what do you think the error could be?

yes it started and failed every time I run the .exe file. I don't know what's wrong, target file is util.js but in error message it shows it's looking for util without extension. maybe that's the problem!

[leafac]

yes it started and failed every time I run the .exe file.

So it seems that the application crashes right as it’s about to start, not after having run for a while, right?

I think there may be a problem in how you’re packaging the application to begin with. Can you share:

1. How you usually start the application, before packaging.
2. How you’re invoking caxa to package the application.

I don't know what's wrong, target file is util.js but in error message it shows it's looking for util without extension. maybe that's the problem!

Maybe. Judging by the error message you’re using ESM, and I believe that unlike CommonJS, ESM insist on having a file extension.

[saostad]

yes it does. so what's the solution?

[leafac]

@saostad Your application must work when use node on the command line before you can package it with caxa. I think the solution of your problem is to add a .js extension to some import somewhere. But I’m afraid I can’t help you much further.

[saostad]

@saostad Your application must work when use node on the command line before you can package it with caxa. I think the solution of your problem is to add a .js extension to some import somewhere. But I’m afraid I can’t help you much further.

I am using typescript to generate my .js file and as far as I know, typescript won't do that.

[maxb2]

I'm not so sure that it is a Typescript error. One of our users had the node binary missing. https://github.com/dungeon-revealer/dungeon-revealer/issues/1545#issuecomment-962463780

C:\Users\dnd\dungeonrevealer\dungeon-revealer-windows-v1.16.0> .\dungeon-revealer-windows.exe
2021/11/06 09:40:42 caxa stub: Failed to run command: exec: "C:\\Users\\dnd\\AppData\\Local\\Temp/caxa/applications/dungeon-revealer-windows\\3bn3q6shhl/0/node_modules/.bin/node": file does not exist

[leafac]

@maxb2: Are you answering to @saostad’s issue? If so, I think these are different issues, because this error message is coming from Node.js, so the node executable was found.

As to the error you reported, could it be the missing .exe at the end?

[maxb2]

All the errors that I'm seeing are missing files in the Windows temporary directory. The executable is extracted and runs correctly at the first run, some time passes, and a subsequent run of the executable complains that a file is missing (so far either server-build\index.js or node_modules/.bin/node). We can fix this broken state by deleting the temporary directory and doing a new extraction. So, it's not an issue with our project. The extracted directory is changing over time.

[gkinsman]

Hi there! I just came across this problem in a project. Is there a way to run a script to clean the temp folder before unpacking? I'd like to keep the single executable deploy if possible, but it looks like the temp folder isn't a very safe place!

For some extra data:

• Running an executable from different locations yields different temp folders.
• After a reboot, each of them stopped working, with different missing files.

Would a more appropriate default be a subfolder in %APPDATA% or %LOCALAPPDATA%? e.g. C:\Users\george\AppData\Roaming\MyElectronApp or C:\Users\george\AppData\Local\MyElectronApp Then it would be untouched by any cleanup process.

[leafac]

@gkinsman That’s the direction we’re heading: Using the AppData directory instead of temporary directories. There’s some discussion about it here: https://github.com/leafac/caxa/issues/20

Would you be able to sketch a pull request?

[SrZorro]

I'm not so sure that it is a Typescript error. One of our users had the node binary missing. dungeon-revealer/dungeon-revealer#1545 (comment)

C:\Users\dnd\dungeonrevealer\dungeon-revealer-windows-v1.16.0> .\dungeon-revealer-windows.exe
2021/11/06 09:40:42 caxa stub: Failed to run command: exec: "C:\\Users\\dnd\\AppData\\Local\\Temp/caxa/applications/dungeon-revealer-windows\\3bn3q6shhl/0/node_modules/.bin/node": file does not exist

My Windows CI failed 3 times with missing node_modules/.bin/node when some time pases from last usage of the CI running the exe.

As a workarround now my CI runs the exe like this: $env:TMP="$PWD\.tmp"; .\my-binary.exe, so it gets unpacked to the working directory inside a .tmp directory that its not going to get tampered by the OS cleanup policies.

There is also the workarround of removing the tmp caxa every time, but that adds the overhead of extracting each time.

[d4ncer]

Our team has this identical issue on macOS and I cannot figure out why 🤔

  throw err;
  ^

Error: Cannot find module '/var/folders/yq/xjb90v7s60196qt0qdqpx61h0000gn/T/caxa/applications/mcli/zgwhljfzhy/0/index.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:939:15)
    at Module._load (node:internal/modules/cjs/loader:780:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
    at node:internal/main/run_main_module:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

This occurs whenever I log out or restart my computer. The folders all correctly exist, but all the files are gone, which I understand will signal to the stub that it does not need to re-extract.

Anyone else facing this on mac?

More system info:

system_profiler SPSoftwareDataType SPHardwareDataType


Software:

    System Software Overview:

      System Version: macOS 12.4 (21F79)
      Kernel Version: Darwin 21.5.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: [REDACTED]
      User Name: [REDACTED]
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 8 days 16:43

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro17,1
      Chip: Apple M1
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 16 GB
      System Firmware Version: 7459.121.3
      OS Loader Version: 7459.121.3
      Serial Number (system): [REDACTED]
      Hardware UUID: [REDACTED]
      Provisioning UDID: [REDACTED]
      Activation Lock Status: Enabled

[leafac]

@d4ncer Oh, that’s so weird! The operating system is cleaning the files in the temporary directory but keeping the directory structures? I didn’t know that was a thing…

You’re right, caxa is looking at the directory structure to determine if it needs to extract the project. This is so that caxa doesn’t need to know much about your project itself.

Perhaps this will be fixed by https://github.com/leafac/caxa/issues/20 ?

[d4ncer]

@d4ncer Oh, that’s so weird! The operating system is cleaning the files in the temporary directory but keeping the directory structures? I didn’t know that was a thing…

Yeah, it's really odd! I'm going to now invest on the macOS side to see why this is happening.

You’re right, caxa is looking at the directory structure to determine if it needs to extract the project. This is so that caxa doesn’t need to know much about your project itself.

Yup, this makes total sense.

Perhaps this will be fixed by #20 ?

It will! Is someone driving this feature?

[leafac]

@d4ncer

It will! Is someone driving this feature?

I don’t think anyone is working on this right now. Do you want to give it a try? 😁

[d4ncer]

I don’t think anyone is working on this right now. Do you want to give it a try? 😁

Will give it a shot :)

[leafac]

@d4ncer

Will give it a shot :)

Hello 👋

Did you have a chance to start working on https://github.com/leafac/caxa/issues/20 ? How’s that going?

It seems that @gkinsman is also interested in giving it a shot, so it’d be nice for us to sync our efforts…

[d4ncer]

@leafac Totally dropped the ball on this, sorry :( Glad someone else is picking it up though! Happy to help support @gkinsman or @maxb2 with testing.

[leafac]

Hi y’all,

Thanks for using caxa and for the conversation here.

I’ve been thinking about the broad strategy employed by caxa and concluded that there is a better way to solve the problem. It no longer extracts your application into a temporary directory, so it shouldn’t run into the issues we talked about here. Can you please give it a try and report back?

It’s a different enough approach that I think it deserves a new name, and it’s part of a bigger toolset that I’m building, which I call Radically Straightforward · Package.

I’m deprecating caxa and archiving this repository. I invite you to continue the conversation in Radically Straightforward’s issues.

Best.