[Bug] Nx fails to cache all the logs generated by @nxrocks/nx-spring-boot

[tschaffter]

Plugin Name

@nxrocks/nx-spring-boot

Plugin Version

4.1.0

Nx Version

13.8.1

Expected Behaviour

Running an nx-cacheable task twice should render the same logs in the terminal when using the executor @nxrocks/nx-spring-boot:*.

Actual Behaviour

Logs generated by @nxrocks/nx-spring-boot:build are not cached.

Steps to reproduce the behaviour

Using the executor @nxrocks/nx-spring-boot:build

Consider this Nx task:

    "build": {
      "executor": "@nxrocks/nx-spring-boot:build",
      "options": {
        "root": "libs/shared/data-access-java"
      }
    },

1. Clear the nx cache: nx clear-cache
2. Run the task

$ nx run shared-data-access-java:build

> nx run shared-data-access-java:build

Executing command: ./mvnw package 
Warning: JAVA_HOME environment variable is not set.
plop
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------< org.sagebionetworks.challenge:data-access-java >-----------
[INFO] Building data-access-java 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ data-access-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ data-access-java ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ data-access-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /workspaces/challenge-registry/libs/shared/data-access-java/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ data-access-java ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ data-access-java ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ data-access-java ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.823 s
[INFO] Finished at: 2022-06-26T16:52:49Z
[INFO] ------------------------------------------------------------------------

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project shared-data-access-java (2s)

3. Run the task again:

$ nx run shared-data-access-java:build

> nx run shared-data-access-java:build  [existing outputs match the cache, left as is]

Executing command: ./mvnw package 

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project shared-data-access-java (15ms)
 
   Nx read the output from the cache instead of running the command for 1 out of 1 tasks.

The logs generated by the Maven wrapper have not been captured by Nx.

Building using Maven directly (without using @nxrocks/nx-spring-boot)

Consider this Nx task that builds the same project as above:

    "build": {
      "executor": "@nrwl/workspace:run-commands",
      "options": {
        "command": "./mvnw package",
        "cwd": "libs/shared/data-access-java"
      }
    },

1. Clear the nx cache: nx clear-cache
2. Run the task

$ nx run shared-data-access-java:build

> nx run shared-data-access-java:build

Warning: JAVA_HOME environment variable is not set.
plop
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------< org.sagebionetworks.challenge:data-access-java >-----------
[INFO] Building data-access-java 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ data-access-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ data-access-java ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ data-access-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /workspaces/challenge-registry/libs/shared/data-access-java/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ data-access-java ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ data-access-java ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ data-access-java ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.761 s
[INFO] Finished at: 2022-06-26T16:56:43Z
[INFO] ------------------------------------------------------------------------

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project shared-data-access-java (2s)

3. Run the task again.

$ nx run shared-data-access-java:build

> nx run shared-data-access-java:build  [existing outputs match the cache, left as is]

Warning: JAVA_HOME environment variable is not set.
plop
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------< org.sagebionetworks.challenge:data-access-java >-----------
[INFO] Building data-access-java 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ data-access-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ data-access-java ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ data-access-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /workspaces/challenge-registry/libs/shared/data-access-java/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ data-access-java ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ data-access-java ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ data-access-java ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.761 s
[INFO] Finished at: 2022-06-26T16:56:43Z
[INFO] ------------------------------------------------------------------------

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project shared-data-access-java (16ms)
 
   Nx read the output from the cache instead of running the command for 1 out of 1 tasks.

This time the logs of the cached task have been properly captured by Nx.

Thanks!

[tinesoft]

Hi @tschaffter

Thanks for using the plugin and for reporting the issue!

I noticed something similar (build not being cached) live during a talk... but back then, I thought it was a just "demo effect"...

Anyway, can you share your nx.json and project.json please?

[tschaffter]

• https://github.com/Sage-Bionetworks/challenge-registry/blob/main/nx.json
• Spring Boot Library project,json
• Spring Boot App project.json

[tinesoft]

Hi @tschaffter Can you update to https://github.com/tinesoft/nxrocks/releases/tag/nx-spring-boot%2Fv5.0.0 and tell me if you're still having this issue?

[tschaffter]

Sure! I will test that once the issue affecting v5.0.0 is fixed.

[tschaffter]

@tinesoft The issue is still there. The first command below shows the output of a build. The second command does the same but uses nx cache and the terminal output is not printed.

vscode@ade7130aff68:/workspaces/challenge-registry$ nx build challenge-service-registry --skip-nx-cache

> nx run challenge-service-registry:build

Executing command: ./mvnw package 
[INFO] Scanning for projects...
[INFO] 
[INFO] ------< org.sagebionetworks.challenge:challenge-service-registry >------
[INFO] Building challenge-service-registry 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ challenge-service-registry ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ challenge-service-registry ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ challenge-service-registry ---
[INFO] Not copying test resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ challenge-service-registry ---
[INFO] Not compiling test sources
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ challenge-service-registry ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ challenge-service-registry ---
[INFO] Building jar: /workspaces/challenge-registry/apps/challenge-service-registry/target/challenge-service-registry-0.0.1-SNAPSHOT.jar
[INFO] 
[INFO] --- spring-boot-maven-plugin:2.7.0:repackage (repackage) @ challenge-service-registry ---
[INFO] Replacing main artifact with repackaged archive
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.702 s
[INFO] Finished at: 2022-08-22T23:46:19Z
[INFO] ------------------------------------------------------------------------

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project challenge-service-registry (6s)
 
   See Nx Cloud run details at https://nx.app/runs/AgQukKOovrT

vscode@ade7130aff68:/workspaces/challenge-registry$ nx build challenge-service-registry                

> nx run challenge-service-registry:build  [existing outputs match the cache, left as is]

Executing command: ./mvnw package 

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project challenge-service-registry (34ms)
 
   Nx read the output from the cache instead of running the command for 1 out of 1 tasks.
 
   See Nx Cloud run details at https://nx.app/runs/XB0Y2nifQAO

[tschaffter]

Do you know whether it's an Nx issue or if it's a plugin issue, for example the plugin may print terminal output in a way that prevent Nx from caching it?

[tinesoft]

It's hard to tell...

• Running a nx build command on a pristine generated Spring-Boot application on Nx workspace < 14 used to work fine
• Running the same command on Nx workspace >=14 has the issue... BUT sometimes it works fine too !
• Currently, running the command on latest version of the plugin (v5.0.1), in a bash terminal within VSCode works fine.

It looks like there is something in the command output that kind of invalidates the Nx caches... but I don"t know what... Maybe it's also linked to the length of the generated output?

One difference that I noticed that may be relevant, is that built-in run-command executor, uses a special option "outputCapture": "pipe", maybe something I will continue investigating...

On what terminal, OS, are you running the build command?

[tschaffter]

On what terminal, OS, are you running the build command?

• VS Code terminal v1.70.2

• Debian dev container

  $ lsb_release -a
  No LSB modules are available.
  Distributor ID: Debian
  Description:    Debian GNU/Linux 11 (bullseye)
  Release:        11
  Codename:       bullseye
  

• /bin/bash shell

[tinesoft]

Ok, I will fork your repo. and try to debug/reproduce from there

[github-actions[bot]]

:tada: This issue has been resolved in version @nxrocks/common-v1.2.2 :tada:

The release is available on:

• GitHub release
• npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

[github-actions[bot]]

:tada: This issue has been resolved in version @nxrocks/nx-spring-boot-v5.0.2 :tada:

The release is available on:

• GitHub release
• npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

[tinesoft]

FYI, as I suspected, that option did the trick:

One difference that I noticed that may be relevant, is that built-in run-command executor, uses a special option "outputCapture": "pipe", maybe something I will continue investigating...

[tschaffter]

I confirm that upgrading to the latest release of the plugin fixes the issue for tasks executed with Maven and Gradle.

[tinesoft]

I'm happy to hear it! Thanks for your feedback and for reporting the issue initially.

Happy coding

[github-actions[bot]]

:tada: This issue has been resolved in version @nxrocks/nx-melos-v1.0.0 :tada:

The release is available on:

• GitHub release
• npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

[github-actions[bot]]

:tada: This issue has been resolved in version 1.0.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[github-actions[bot]]

:tada: This issue has been resolved in version 1.0.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[github-actions[bot]]

:tada: This issue has been resolved in version 1.0.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[github-actions[bot]]

:tada: This issue has been resolved in version 1.0.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket: