"pip install" step of README instructions fails, requires "yarn install"

[gagern]

I was trying to follow the README but it failed for me:

https://github.com/jupyter-widgets/widget-ts-cookiecutter/blob/94f9d384331150655c18a08d94307280c32adedd/README.md?plain=1#L50-L54

$ pip install -e ".[test,examples]"
Obtaining file://${PWD}
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [65 lines of output]
      INFO:hatch_jupyter_builder.utils:Running jupyter-builder
      INFO:hatch_jupyter_builder.utils:Building with hatch_jupyter_builder.npm_builder
      INFO:hatch_jupyter_builder.utils:With kwargs: {'path': '.', 'build_cmd': 'build:prod'}
      INFO:hatch_jupyter_builder.utils:Installing build dependencies with npm.  This may take a while...
      INFO:hatch_jupyter_builder.utils:> ${HOME}/mambaforge/envs/${PROJECT}-dev/bin/npm install
      npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-url#deprecated
      npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
      npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
      npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-resolve#deprecated
      npm WARN deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
      npm WARN deprecated [email protected]: Use your platform's native performance.now() and performance.timeOrigin.
      npm WARN deprecated [email protected]: some dependency vulnerabilities fixed, support for node < 10 dropped, and newer ECMAScript syntax/features added
      npm WARN deprecated [email protected]: This package is no longer supported. It's now a built-in Node module. If you've depended on crypto, you should switch to the one that's built-in.
      
      added 1104 packages, and audited 1105 packages in 54s
      
      122 packages are looking for funding
        run `npm fund` for details
      
      found 0 vulnerabilities
      INFO:hatch_jupyter_builder.utils:> ${HOME}/mambaforge/envs/${PROJECT}-dev/bin/npm run build:prod
      
      > ${PROJECT}@0.1.0 build:prod
      > yarn run build:lib && yarn run build:nbextension && yarn run build:labextension
      
      Internal Error: ${PROJECT}@workspace:.: This package doesn't seem to be present in your lockfile; run "yarn install" to update the lockfile
          at P0.getCandidates (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:435:5145)
          at yf.getCandidates (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:391:1264)
          at ${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:439:7693
          at of (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:390:11070)
          at ge (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:439:7673)
      Traceback (most recent call last):
        File "${HOME}/mambaforge/envs/${PROJECT}-dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 167, in prepare_metadata_for_build_editable
          hook = backend.prepare_metadata_for_build_editable
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "${HOME}/mambaforge/envs/${PROJECT}-dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "${HOME}/mambaforge/envs/${PROJECT}-dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "${HOME}/mambaforge/envs/${PROJECT}-dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 176, in prepare_metadata_for_build_editable
          whl_basename = build_hook(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-5gzrg0oe/overlay/lib/python3.11/site-packages/hatchling/build.py", line 78, in build_editable
          return os.path.basename(next(builder.build(wheel_directory, ['editable'])))
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-5gzrg0oe/overlay/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 150, in build
          build_hook.initialize(version, build_data)
        File "/tmp/pip-build-env-5gzrg0oe/normal/lib/python3.11/site-packages/hatch_jupyter_builder/plugin.py", line 83, in initialize
          raise e
        File "/tmp/pip-build-env-5gzrg0oe/normal/lib/python3.11/site-packages/hatch_jupyter_builder/plugin.py", line 78, in initialize
          build_func(self.target_name, version, **build_kwargs)
        File "/tmp/pip-build-env-5gzrg0oe/normal/lib/python3.11/site-packages/hatch_jupyter_builder/utils.py", line 116, in npm_builder
          run(npm_cmd + ["run", build_cmd], cwd=str(abs_path))
        File "/tmp/pip-build-env-5gzrg0oe/normal/lib/python3.11/site-packages/hatch_jupyter_builder/utils.py", line 227, in run
          return subprocess.check_call(cmd, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "${HOME}/mambaforge/envs/${PROJECT}-dev/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['${HOME}/mambaforge/envs/${PROJECT}-dev/bin/npm', 'run', 'build:prod']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

The key part here is presumably the statement

This package doesn't seem to be present in your lockfile; run "yarn install" to update the lockfile

The stack trace is more of a red herring, with the Python output distracting from the npm output. So I did a yarn install and repeated the pip install. This time it made a tiny bit more progress:

      INFO:hatch_jupyter_builder.utils:> ${HOME}/mambaforge/envs/${PROJECT}-dev/bin/npm run build:prod
      
      > ${PROJECT}@0.1.0 build:prod
      > yarn run build:lib && yarn run build:nbextension && yarn run build:labextension
      
      Internal Error: ${PROJECT}@workspace:.: This package doesn't seem to be present in your lockfile; run "yarn install" to update the lockfile
          at P0.getCandidates (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:435:5145)
          at yf.getCandidates (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:391:1264)
          at ${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:439:7693
          at of (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:390:11070)
          at ge (${HOME}/mambaforge/envs/${PROJECT}-dev/bin/yarn.js:439:7673)

I did a manual yarn install followed by npm run build:prod and then my pip install finally succeeded. It would be nice to have the docs or the code updated so that this just works.

[ianhi]

I think that something must have changed as this definitely didn't used to be the case

[maartenbreddels]

I think I worked around this in bqplot by requiring yarn<=1

[ianhi]

The second error seems to be resolved by updating the minimum hatch build to 1.6.0