TypeError: workersReadyResolve is not a function error while compiling

[dfstio]

I have seen many times errors while compiling different contracts in the same worker:

Error in task TypeError: workersReadyResolve is not a function
    at startWorkers (o1js/dist/node/index.cjs:3741:3)
    at async initThreadPool (o1js/dist/node/index.cjs:3711:3)
    at async withThreadPool (o1js/dist/node/index.cjs:3678:3)
    at async prettifyStacktracePromise (o1js/dist/node/index.cjs:1934:12)
    at async compileProgram (o1js/dist/node/index.cjs:9689:60)
    at async Object.compile (o1js/dist/node/index.cjs:9520:67)

Sometimes the worker hangs out while compiling. To reproduce this effect, the worker should run for several hours and compile few different contracts. The workers I use have 10240 MB of memory.

[dfstio]

Maybe it happens because wasm errors occur and are not being handled https://github.com/o1-labs/o1js-bindings/blob/main/js/node/node-backend.js#L85-L90

[dfstio]

I was able to get rid of this error by having dedicated workers for Devnet and Zeko. Seems like the problem was in calling in the same worker first Mina.setActiveInstance for Devnet, then Mina.setActiveInstance for Zeko, then Mina.setActiveInstance for Devnet again etc.

@mitschabaude should Mina.setActiveInstance clean static variables and global state when it is being called?

[dfstio]

Again, I saw this error today after many proofs were created in the same worker for the same chain (devnet).

[SpaceManiac]

I was able to trigger the same error by importing two copies of o1js due to using npm link. The second copy takes over globalThis.startWorkers and gets called back by WASM despite the first copy being the only one that has actually initialized workersReadyResolve.

import { ZkProgram } from 'o1js';
import 'o1js_duplicate';  // In reality, an indirect/accidental duplication

const NoopProgram = ZkProgram({
  name: 'NoopProgram',

  methods: {
    proof: {
      privateInputs: [],
      async method(): Promise<void> {
      }
    }
  }
});
await NoopProgram.compile();

TypeError: workersReadyResolve is not a function
    at startWorkers (o1js_duplicate/src/bindings/js/node/node-backend.js:125:3)
    at initThreadPool (o1js/src/bindings/js/node/node-backend.js:86:3)
    at withThreadPool (o1js/src/bindings/js/node/node-backend.js:50:3)
    at prettifyStacktracePromise (o1js/src/lib/util/errors.ts:138:12)
    at compileProgram (o1js/src/lib/proof-system/zkprogram.ts:958:5)
    at Object.compile (o1js/src/lib/proof-system/zkprogram.ts:648:48

[dfstio]

This problem will be even bigger on the mainnet: For example, o1js 1.3 breaks the verification keys of one of my contracts due to the changes in the wrap domain size. If I want to add the new version of my contract to my frontend and backend and use the last version of o1js because of some new o1js features, for example, o1js 1.5, I need to have two o1js versions—one (o1js 1.2) for the old contract and one (o1js 1.5) for the new contract as I cannot simply stop using on mainnet the old contract that has users, and the upgrading of the verification keys is not always possible.

[mitschabaude]

I think this is valid and should be fixed as follows: o1js should detect when it's being used twice, right within its loading cycle, and throw a clear error message in that case

[45930]

@mitschabaude

How would you suggest to use libraries in that case? I've always had this issue with o1js-pack, and an easy fix is to copy-paste the source code from that library into my project. That gets around the issue of the library requiring o1js and the projects it's in also requiring o1js. But it's not ideal from a community standpoint, and it certainly doesn't scale to the point where there are dozens of solid community libraries available.

I wonder if libraries will need to bundle themselves differently, using the o1js types for checking, but ultimately building themselves with a host's copy of o1js, rather than building themselves with their own dependency on o1js.

[mitschabaude]

@45930 that's exactly why we recommend having o1js as peer-dependency in your package.json, except in the final UI project.

Re bundling, you shouldn't bundle libraries in general (o1js in the Node.js version is also not bundled)

A peer-dependency means that node/bundlers will resolve it to a dependency of the top-level (UI) project, so you don't get it twice.

The entire React ecosystem operates (or at least used to operate when I was there) under the same restriction and people were always using React as a peer dependency https://legacy.reactjs.org/warnings/invalid-hook-call-warning.html#duplicate-react