12.0.0-alpha.13: Pervasives not found when using pnpm

[hackwaly]

version: 12.0.0-alpha.13 platform: macOS arm64

demo https://gist.github.com/hackwaly/bae3945b92170e79b9130bf1b6019209

pnpm rewatch watch
[1/7] 📦 Built package tree in 0.00s
[2/7] 👀 Found source files in 0.00s
[3/7] 📝 Read compile state 0.00s
[4/7] 🧹 Cleaned 0/2 0.00s
[5/7] 🧱 Parsed 0 source files in 0.00s
[6/7] 🌴 Collected deps in 0.00s
[7/7] ❌ Compiled 2 modules in 0.01s

  We've found a bug for you!
  command line

  The module or file Pervasives can't be found.
  - If it's a third-party dependency:
    - Did you add it to the "bs-dependencies" or "bs-dev-dependencies" in rescript.json?
  - Did you include the file's directory to the "sources" in rescript.json?
  


  We've found a bug for you!
  command line

  The module or file Pervasives can't be found.
  - If it's a third-party dependency:
    - Did you add it to the "bs-dependencies" or "bs-dev-dependencies" in rescript.json?
  - Did you include the file's directory to the "sources" in rescript.json?

Thank you for filing! Check list:

• [x] Is it a bug? Usage questions should often be asked in the forum instead.
• [x] Concise, focused, friendly issue title & description.
• [x] A minimal, reproducible example.
• [x] OS and browser versions, if relevant.
• [ ] Is it already fixed in master? Instructions

[nojaf]

Thanks for reporting this. I can reproduce this. Both rewatch and bsb can't find Pervasives.src. I believe this is related to pnpm.

[hackwaly]

I believe the root cause is here: https://github.com/rescript-lang/rescript/blob/a375a4780ec7531c628a6f5fc915645a1387e50a/rewatch/src/build/compile.rs#L474-L477

It wrongly assumed that include_dir is just "../ocaml".

I think we should change this https://github.com/rescript-lang/rescript/blob/a375a4780ec7531c628a6f5fc915645a1387e50a/cli/rewatch.js#L10 to pass the "--rescript-home" dir to rewatch, and let rewatch to use that to generate argument of -I, as we split rescript bin to dedicated package.

FYI: https://github.com/rescript-lang/rescript/blob/a375a4780ec7531c628a6f5fc915645a1387e50a/rewatch/src/build/compile.rs#L490-L493

[nojaf]

Thanks for doing some investigation here @hackwaly! @jfrolich any thoughts?

[jfrolich]

@hackwaly that is the path of the compiler assets fo the current package, for the dependencies we use an absolute path based on the node modules directory. The pervasive module is the standard library and that is something the compiler always includes (and is included by the compiler not by the build system). Are you sure it works in bsb. Otherwise we have to fix it in the compiler.

[hackwaly]

I'm sure it not works with bsb.

[cknitt]

The paths for bsc.exe and Pervasives.cmi look like this for npm:

./node_modules/@rescript/darwin-arm64/bin/bsc.exe
./node_modules/rescript/lib/ocaml/Pervasives.cmi

but like this for pnpm:

./node_modules/.pnpm/@[email protected]/node_modules/@rescript/darwin-arm64/bin/bsc.exe
./node_modules/.pnpm/[email protected]/node_modules/rescript/lib/ocaml/Pervasives.res

The logic at

https://github.com/rescript-lang/rescript/blob/d70c7fa8dbfd799303154e3f254f8cd6d99312bd/compiler/ext/config.ml#L7

looks for <directory of bsc.exe>/../../../rescript/lib/ocaml which works for npm, but not pnpm.

[cknitt]

@hackwaly Can you try with npm i https://pkg.pr.new/rescript-lang/rescript@bcc65cb?

[hackwaly]

@hackwaly Can you try with npm i https://pkg.pr.new/rescript-lang/rescript@bcc65cb?

It doesn't work either. I think I should install the PR version of @rescript/darwin-arm64. But how do I do that?

[cknitt]

It doesn't work either. I think I should install the PR version of @rescript/darwin-arm64. But how do I do that?

You do not need to do that, it is installed automatically as a dependency of the rescript package.

The following works for me:

• Download and extract your example project
• pnpm remove @rescript/darwin-arm64
• pnpm i https://pkg.pr.new/rescript-lang/rescript@bcc65cb
• pnpm rescript build (rewatch is now the default, so it's just "rescript" instead)

Can you confirm?

[sgrove]

This still seems to be an issue for me using 12.0.0-beta.7 (can't find the Pervasives modules repeated many times). Maybe one wrinkle is that it's a pnpm workspace with both backend/frontend in rescript, each with their own package.json - not sure if that matters or not, just sharing in case it's useful!

Happy to share more details, of course!

[cknitt]

@sgrove I'll soon pick up the work in #7637 again.

It would be great if you could share a repro.

[alex35mil]

I'm not sure if it's related, but I tried to upgrade my monorepo to v12, but it seems like rescript can't find hoisted dependencies:

[1/7] 📦 Building package tree...[1/2] ❌  Error building package tree. The package "@rescript/core" is not found (are node_modules up-to-date?)...
[1/2] ❌  Error building package tree. The package "@rescript/react" is not found (are node_modules up-to-date?)...
[1/2] ❌  Error building package tree. The package "rescript-logger" is not found (are node_modules up-to-date?)...
[1/2] ❌  Error building package tree. The package "rescript-webapi" is not found (are node_modules up-to-date?)...
ERROR:
We could not build package tree reading dependency 'rescript-webapi', at path '/Users/Alex/Dev/psc/clients/web/lib'. Error: The package "rescript-webapi" is not found (are node_modules up-to-date?)...
ERROR:
We could not build package tree reading dependency '@rescript/react', at path '/Users/Alex/Dev/psc/clients/web/lib'. Error: The package "@rescript/react" is not found (are node_modules up-to-date?)...
ERROR:
We could not build package tree reading dependency '@rescript/core', at path '/Users/Alex/Dev/psc/clients/web/lib'. Error: The package "@rescript/core" is not found (are node_modules up-to-date?)...

I use npm workspaces.

[alex35mil]

After digging in the v12 for a bit, it seems like I need to have a root rescript.json in a monorepo.

[alex35mil]

Can't find any documentation regarding it, and simply adding rescript.json with dependencies to the root doesn't help.

[nojaf]

Hello @alex35mil, thank you for raising this.

We discussed this on a call this week and are experiencing the same problem in our own repository. (meeting notes)

We will ensure hoisted dependencies are discoverable again and perform the same final resolution attempts used in the old build system. For now, please use rescript legacy.

There is no documentation yet because this is still in flux. I began drafting some documentation, but it is no longer accurate.

After digging in the v12 for a bit, it seems like I need to have a root rescript.json in a monorepo.

Yes. The intention was to make resolution stricter so packages are resolved from the monorepo root only when there is a proper setup. This will change in future betas to be less strict and more compatible with the legacy build tool.

Could you outline your project layout? Please share your folder structure and which package manager you use.

[alex35mil]

Hey @nojaf, duplicating the structure here:

node_modules/
web/
    - lib/
    - apps/
        - app1/
        - app2/

I use npm with workspaces.

After I added all monorepo packages as deps, the mentioned issue is gone. But I'm getting Pervasives issue now.

The root rescript.json looks like this:

{
    "name": "psc",
    "sources": [],
    "package-specs": [
        {
            "module": "esmodule",
            "in-source": true
        }
    ],
    "suffix": ".res.mjs",
    "jsx": {
        "version": 4
    },
    "dependencies": [
        "@psc/lib",
        "@psc/platform",
        "@psc/sites"
    ],
    "warnings": {
        "number": "+A"
    }
}

[nojaf]

Could you tell me on the node_modules level where https://www.npmjs.com/package/@rescript/runtime got installed?

[alex35mil]

I install it in the package, but npm hoists it, so it ends up in the root node_modules/@rescript/runtime.

Install command:

npm install --save --save-exact --workspace=clients/web/apps/platform --workspace=clients/web/apps/sites --workspace=clients/web/lib @rescript/runtime

[nojaf]

Hi @hackwaly, @alex35mil and @sgrove,

Can you try https://github.com/rescript-lang/rescript/releases/tag/v12.0.0-beta.11?

This version might still have other issues, but at the very least the runtime should be found and you should no longer see Pervasives not found.

The latest version needs a matching prerelease of the tooling extension (1.65.5).

[hackwaly]

I have tried beta 11, it works well.

But, the vscode extension has an issue: Gotodef on modules from stdlib (e.g. Array) not works.

[sgrove]

I've tried beta 11, and seems to be working for me as well!

I haven't tried the new extension at all however, but thank you for the quick fix!