Add support for Yarn 2 / Yarn 3

[kayb-proxora]

Hello,

Is it possible to use yarn 2 with the plugin?

In normal yarn 2 setup you have to enable yarn 2 via: yarn set version berry

Is there a way to also do this with the plugin?

Thanks & regards,

Kay

[deepy]

Up to this point I thought that yarn was the sane alternative to npm, but the way to change to yarn 2 is bizarre.

I haven't tested this myself, but I think something like this might work:

def setVersion = tasks.register(yarnSetVersion, YarnTask) {
    args = ['set', 'version', 'berry']
}

tasks.named('yarnSetup').configure { finalizedBy setVersion }

You might have to run the yarnSetVersion task once manually

[kayb-proxora]

Thanks for the quick feedback. I tried your suggestion, but it seems that the install task is not triggered now:

Internal Error: react-frontend@workspace:.: This package doesn't seem to be present in your lockfile; try to make an install to update your resolutions
    at J.getCandidates (\react-frontend\.yarn\releases\yarn-2.4.2.cjs:2:353956)
    at n.getCandidates (\react-frontend\.yarn\releases\yarn-2.4.2.cjs:2:342946)
    at \react-frontend\.yarn\releases\yarn-2.4.2.cjs:2:364602
    at Module.w (\react-frontend\.yarn\releases\yarn-2.4.2.cjs:2:424073)
    at C (\react-frontend\.yarn\releases\yarn-2.4.2.cjs:2:364565)

Yarn 2 is also generating different files. It creates a .yarn folder and a .yarnrc.yml file. Maybe the plugin has to handle yarn 2 differently?

[deepy]

yeah it sounds like we're going to have to look into how we can support yarn 2. but their setup seems pretty bizarre to me, I don't understand why they wouldn't either set the version to 2 for yarn 2 or create a yarn2 package if they absolutely need them to be separate and installed at the same time

[kayb-proxora]

Yes, I agree. The way to use yarn 2 is very strange.

Would be great if you can support yarn 2 in a future release. :)

[boris-petrov]

For starters, Yarn 2 support with the backwards-compatible nodeLinker option could be implemented. As you see from the migration guide, it should be pretty straight-forward to migrate to Yarn 2 in this way (i.e. not using the new PnP mechanisms). And, according to people on the web, it's time to migrate to Yarn 2. :) So it would be nice to support this.

[xenoterracide]

hmm, +1 though I would prefer the ability to use PnP though this doesn't seem like it should be hard?

[boris-petrov]

So there is Yarn 3 already. I can't seem to keep up with these releases. :smiley: Has anyone tried that with gradle-node-plugin?

[ericparton]

So I have some good news; if you run that weird yarn set version command, it very explicitly says exactly what it's doing:

YN0000: Retrieving https://repo.yarnpkg.com/3.1.0/packages/yarnpkg-cli/bin/yarn.js
YN0000: Saving the new release in .yarn/releases/yarn-3.1.0.cjs 

If that's all that's necessary, it should be possible to replicate those two steps in the plugin

[strawberry-choco]

Any updates on this? Yarn 3 is extremely awesome, and we would love to use it. :)

[hoanvh]

I'm using Yarn 3 with this plugin already, without any problem until now:

plugins {
    id 'com.github.node-gradle.node' version '3.5.0'
}

node {
    download = true
    version = '16.18.1'         // NodeJS version
    yarnVersion = '1.22.19'     // Yarn 1 version, we will upgrade to Yarn 3 later
}

task yarnUpgrade(type: YarnTask, dependsOn: 'yarnSetup') {
    args = ['set', 'version', 'stable']     // maybe with `yarn set version 3.x` also
}

task yarnInstall(type: YarnTask, dependsOn: 'yarnUpgrade') {
    args = ['install', '--immutable']
}

task yarnBuild(type: YarnTask, dependsOn: 'yarnInstall') {
    args = ['run', 'build', "--mode=${project.getProperties().getOrDefault("mode", "local")}"]
    environment = [
            BUILD_VERSION: project.version,
            BUILD_TIME: "${System.currentTimeMillis()}",
    ]
}

task build(dependsOn: 'yarnBuild') {
    //
}

[strawberry-choco]

Yes, correct. It works, but not nice since we are relying on yarn 1 itself. Can't we have native yarn berry support?

[muenchto]

Hi @hoanvh , thanks for this possible solution. However, now I have the problem that yarn 2+ will actually upgrade my dependencies in the yarn.lock, within the ranges defined in the package.lock (like ^1.0.2). Consequently, this will cause failure because of the --immutable argument, that you rightfully pass in your example. Do you have any solution for this problem?

What is strange, that the checked-in yarn.lock was already created with yarn 2+ locally. So why is the yarn 2+ inside the gradle plugin not respecting this existing valid yarn.lock?

Thanks in advance!

[hoanvh]

--immutable flag makes sure that the installation from Yarn will not change our dependencies (name, version, ...). In my case, whenever there is a failure causing by --immutable, I will install packages without that flag first, review the new version and commit it (package.lock / yarn.lock). In your case, I guess the version of yarn 2+ before & after does not match. Inside lock file, some of packages will be updated whenever there is a new version of Yarn. In my case, I use typescript package, and even I'm using a fixed version, its commit hash will always be changed, whenever I upgrade Yarn version (by yarn set version stable command). And it's normal.

[eekboom]

Extremely sad and also quite surprised that yarn 3 is not supported. 😢 In fact if it helps it would be ok to not support yarn 2, but only versions 1 and >= 3.

[muenchto]

Hi @eekboom I think in the end we got it a bit wrong in this thread. We got yarn 3 in our project running with this gradle plugin!

Basically, yarn 3 is confusing most people, including me. What yarn 3 gives you is not a new yarn on your system but a kind of yarn-executor in your project. That is, you have a system yarn 1.x.x AND a project yarn 3.x.x.

What does this mean for this plugin? Just keep the yarn version parameter with (latest) yarn1 like yarnVersion = "${yarn_v1_version}". This will be your "system yarn". In your frontend project you specify your yarn 3 version anyways, probably in package.json like "packageManager": "[email protected]". All yarn commands that you issue with the gradle plugin will be just passed through to your yarn 3 of the project. Therefore, make sure that you adapt any yarn commands to the new syntax of yarn 3, e.g. yarn install --immutable instead of yarn install --frozen-lockfile. In contrast to @hoanvh posted solution, afaik you dont have to do

task yarnUpgrade(type: YarnTask, dependsOn: 'yarnSetup') {
    args = ['set', 'version', 'stable']     // maybe with `yarn set version 3.x` also
}

Sorry for not passing this info to the thread earlier.. One thing to mention, however, we do not yet use the plug'n'play feature of yarn 3 yet.. maybe there are more problems when doing so.. But unless nobody reports problems or corrects my understanding here, @deepy can maybe clarify this in the documentation.

[deepy]

@muenchto would you (or someone else) be able to create a small example project of this? If not I'll try following the instructions and create one, adding it to the test suite and then updating the documentation

Right now I've been neglecting yarn 2 and 3 as they don't map cleanly into the current model, but the major rework I'm slowly chipping away at should make supporting all yarn versions smooth as butter

I need to get the configuration-cache support properly fixed first, but when that's done I'll return to this

[eekboom]

Thanks a lot for the answer! That is ... weird 😮 and it feels really wrong to specify yarn 1 for the gradle plugin and yarn 3 in package.json. But it works and from a first look it also seems to work with pnp.

[gerardnico]

It does not work for me at all with the gradle-node-plugin version 5.0.0 Even if I set up the version of yarn to 1.22.19, it still downloads the dependencies and does not use yarn 3 specified in the package.json.

> Task :xxxx:yarn
yarn install v1.22.19
[1/4] Resolving packages...
warning @material-ui/[email protected]: Material UI v4 doesn't receive active development since September 2021. See the guide https://mui.com/material-ui/migration/migration-v4/ to upgrade to v5.
warning @material-ui/core > @material-ui/[email protected]: Material UI v4 doesn't receive active development since September 2021. See the guide https://mui.com/material-ui/migration/migration-v4/ to upgrade to v5.
warning react-scripts > [email protected]: this package has been deprecated
warning react-scripts > workbox-webpack-plugin > [email protected]: this package has been deprecated

I have no time to dig into the code, I'm going to use an Exec task for now. For info, yarn3 needs to be installed with corepack as specified in the doc.

[deepy]

I like the approach corepack is going but since it's operating on global state it's a little tricky getting the version right, even more so in CI environments But to get corepack supported there's going to have to be some overhauls, I think it's a big improvement over the solution presented by yarn 2 though (Really only because it promises standardization and can pull the package manager from package.json and also supports pnpm)

[mrclrchtr]

Any updates here?

[deepy]

@mrclrchtr not much unfortunately, there's a workaround provided in https://github.com/node-gradle/gradle-node-plugin/issues/176#issuecomment-1531948622 and I'm still hoping someone will create an example project showcasing it Personally my interest in Yarn stopped when everything started becoming global (the bizarre setup I mentioned previously)

Supporting Yarn 2 or 3 is currently a bit more work than I'm able to invest, my current focus is on supporting external PRs and to make it easier to work on the plugin Refactoring and updating the core of the plugin is going to make it easier to work on this issue

[mrclrchtr]

Ok, thanks for the info and your effort. Then I know that I have to think of an alternative ;)

[deepy]

Just want to stress the point that this is basically only about priorities, and with the recent PR adding bun support I'm now in a state where I have to start addressing the core of the plugin (which is good, because it'll make the code easier to read) So if anyone is in a position where they have the time to work on this I will happily guide you throughout the entire process and do the finishing touches myself

[virjar]

fix corepack symbolicLink after Node unpack, and enable corepack after NodeSetupTask

tasks.withType(NodeSetupTask::class.java).configureEach {
    doLast {
        val nodeExtension = NodeExtension[project]
        val variantComputer = VariantComputer()

        val isWindows = nodeExtension.resolvedPlatform.get().isWindows()

        // fix corepack symbolicLink
        fun computeCorepackScriptFile(nodeDirProvider: Provider<Directory>): Provider<String> {
            return nodeDirProvider.map { nodeDir ->
                if (isWindows) nodeDir.dir("node_modules/corepack/dist/corepack.js").asFile.path
                else nodeDir.dir("lib/node_modules/corepack/dist/corepack.js").asFile.path
            }
        }

        val nodeDirProvider = nodeExtension.resolvedNodeDir
        val nodeBinDirProvider = variantComputer.computeNodeBinDir(nodeDirProvider, nodeExtension.resolvedPlatform)
        val nodeBinDirPath = nodeBinDirProvider.get().asFile.toPath()
        val corepackScript = nodeBinDirPath.resolve("corepack")
        val scriptFile =
            computeCorepackScriptFile(nodeDirProvider)
        if (Files.deleteIfExists(corepackScript)) {
            Files.createSymbolicLink(corepackScript, nodeBinDirPath.relativize(Paths.get(scriptFile.get())))
        }

        val yarnDir = variantComputer.computeYarnDir(nodeExtension).get()
        val dirPath = if (isWindows) yarnDir else yarnDir.dir("bin")

        val nodeExecutable = nodeBinDirPath.resolve("node")
        mkdir(dirPath)
        exec {
            // actually YarnSetup execute here
            commandLine(nodeExecutable, corepackScript, "enable", "--install-directory", dirPath)
        }
    }
}


tasks.withType(YarnSetupTask::class.java).configureEach {
    enabled = false
}

[ptitjes]

@virjar do you have an example project using gradle-node-plugin with yarn 4.5.0, by any chance? I am trying to apply your recipe but I am completely failing...

[virjar]

@ptitjes https://github.com/yint-tech/atom/blob/main/frontend/build.gradle.kts