[MDEP-757] bug with "non-test scoped test only dependencies found"

[jira-importer]

Henning Schmiedehausen opened MDEP-757 and commented

Starting with 3.2.0, I see this problem in one of my project modules:

[INFO] --- maven-dependency-plugin:3.2.0:analyze-only (basepom.default) @ foundation ---
[WARNING] Non-test scoped test only dependencies found:
[WARNING] com.fasterxml.jackson.core:jackson-databind:jar:2.12.2:compile

This is not correct:

% cd lib/foundation

% grep -r databind src/main

src/main/java/dev/data/DataSet.java:import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
src/main/java/dev/data/DataSet.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;
src/main/java/dev/data/WriteableLocator.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;
src/main/java/dev/data/ObjectMetadata.java:import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
src/main/java/dev/data/ObjectMetadata.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;
src/main/java/dev/data/DataDefinition.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;
src/main/java/dev/data/ObjectAttribute.java:import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
src/main/java/dev/data/ObjectAttribute.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;
src/main/java/dev/data/ObjectDefinition.java:import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
src/main/java/dev/data/ObjectDefinition.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;
src/main/java/dev/data/ImmutableDataObject.java:import com.fasterxml.jackson.databind.annotation.JsonSerialize;

So this dependency is used all over the place in the main sources (which is why it can not be in test scope). However, the plugin no longer (it did in 3.1.2) recognize it (all the uses are just annotations, so I guess that is the problem) and raises this error.

---

Affects: 3.2.0

Attachments:

• MDEP-757-reproducer-2-rec.zip (14.57 kB)

Issue Links:

• MDEP-753 Non-test dependency reported as Non-test scoped test only dependency

Remote Links:

• GitHub Pull Request #82

11 votes, 20 watchers

[jira-importer]

Henning Schmiedehausen commented

minimal project to reproduce the issue: https://github.com/hgschmie/mdep757

 

 

[jira-importer]

Henning Schmiedehausen commented

Could be a duplicate of MDEP-753; however this issue specifically refers to annotations.

[jira-importer]

Slawomir Jaranowski commented

still exists with maven-dependency-analyzer 1.12.0

[jira-importer]

Henning Schmiedehausen commented

Please try 3.3.0 and add 

 <ignoredNonTestScopedDependencies>     <ignoreNonTestScopedDependency>*</ignoreNonTestScopedDependency> </ignoredNonTestScopedDependencies>

 

to the plugin configuration.

[jira-importer]

Roman Huber commented

This still exists with 3.5.0.

I have the following setup:

• My project has a dependency 'A' with a compile scope dependency 'B'
• I'm using classes of 'B' in my tests.
• I have to declare 'B' as a test scope dependency, otherwise analyze fails with 'non-test scoped test only dependencies found'

[jira-importer]

Henning Schmiedehausen commented

use <ignoreAllNonTestScoped>true</ignoreAllNonTestScoped>

 

[jira-importer]

Richard Eckart de Castilho commented

Is there any reason why ignoreAllNonTestScoped is not true by default?

[jira-importer]

Richard Eckart de Castilho commented

I still think this is a bug because the analyzer should be able to find a path from classes included in A to classes included in B and thereby deduce that B is needed in the same scope as A, no?

[jira-importer]

Richard Eckart de Castilho commented

Here is another reproducer.

• Either maven-dependency-plugin complains about

[ERROR] Non-test scoped test only dependencies found:
[ERROR]    example:library:jar:0.0.1-SNAPSHOT:compile

• or if I make the dependency a test dependency, then the compiler complains:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project consumer: Compilation failure
[ERROR] .../root/consumer/src/main/java/consumer/ActualImplementationClass.java:[8,23] cannot access library.LibraryObject
[ERROR]   class file for library.LibraryObject not found

[^MDEP-757-reproducer-2-rec.zip]

It works ok with library as a test as long as ActualImplementationClass is not calling the AbstractBaseClass.getObject() method - but once that method call is added, the compiler fails.

Looking into the analyzer source code, it appears that the analysis only considers surface properties like method signatures, but does not seem to check what is done inside the methods.

[jira-importer]

Richard Eckart de Castilho commented

Analyzing the bytecode of the consumer class, the reference to library.LibraryObject can be seen in the method reference that is in the constant pool.

% javap -c -verbose consumer.ActualImplementationClass
Classfile .../consumer/ActualImplementationClass.class
  Last modified Mar 17, 2023; size 458 bytes
  MD5 checksum bd73f558e3ab51f8c30011d78a7ffe6b
  Compiled from "ActualImplementationClass.java"
public class consumer.ActualImplementationClass extends provider.AbstractBaseClass
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Methodref          #2.#3          // provider/AbstractBaseClass."<init>":()V
   #2 = Class              #4             // provider/AbstractBaseClass
   #3 = NameAndType        #5:#6          // "<init>":()V
   #4 = Utf8               provider/AbstractBaseClass
   #5 = Utf8               <init>
   #6 = Utf8               ()V
   #7 = Methodref          #8.#9          // consumer/ActualImplementationClass.getObject:()Llibrary/LibraryObject;
   #8 = Class              #10            // consumer/ActualImplementationClass
   #9 = NameAndType        #11:#12        // getObject:()Llibrary/LibraryObject;
  #10 = Utf8               consumer/ActualImplementationClass
  #11 = Utf8               getObject
  #12 = Utf8               ()Llibrary/LibraryObject;
  #13 = Utf8               Code
  #14 = Utf8               LineNumberTable
  #15 = Utf8               LocalVariableTable
  #16 = Utf8               this
  #17 = Utf8               Lconsumer/ActualImplementationClass;
  #18 = Utf8               doSomething
  #19 = Utf8               SourceFile
  #20 = Utf8               ActualImplementationClass.java
{
  public consumer.ActualImplementationClass();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method provider/AbstractBaseClass."<init>":()V
         4: return
      LineNumberTable:
        line 5: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   Lconsumer/ActualImplementationClass;
}
SourceFile: "ActualImplementationClass.java"

However, the ConstantPoolParser.getConstantPoolClassReferences( byteCode ) call in org.apache.maven.shared.dependency.analyzer.asm.DependencyClassFileVisitor.visitClass(String, InputStream) does not seem to return that. It only finds [provider/AbstractBaseClass, consumer/ActualImplementationClass].

[jira-importer]

Richard Eckart de Castilho commented

Btw. I can reproduce https://github.com/hgschmie/mdep757 on Java 11 with

<dep.dependency-analyzer.version>1.11.3</dep.dependency-analyzer.version>
<dep.plugin.dependency.version>3.2.0</dep.plugin.dependency.version>

but not with (or higher)

<dep.dependency-analyzer.version>1.13.0</dep.dependency-analyzer.version>
<dep.plugin.dependency.version>3.3.0</dep.plugin.dependency.version>

[jira-importer]

Richard Eckart de Castilho commented

Fix for the second reproducer: https://github.com/apache/maven-dependency-analyzer/pull/82

[jira-importer]

Henning Schmiedehausen commented

you are most likely correct but you need to convince the maintainers of the plugin and the dependency analyzer library that they are not which in my experience is an uphill battle. 

Setting the ignoreAllNonTestScoped flag and moving on with life was my answer to that.

[jira-importer]

Richard Eckart de Castilho commented

Well, there is a PR now and it already has one approval - that's at least a start. In the worst case, I can do an internal release of the fix for those projects where I want/need it. But I do hope that maybe in the not-so-distant future, we might also get a analyzer release including the fix. Once the analyzer is released, it can immediately be used by the dependency plugin by adding it as a dependency to the plugin declaration in the build section of the POM - so no new dependency plugin release is required - only an analyzer release.

[jira-importer]

Richard Eckart de Castilho commented

In my experience the "uphill battle" is typically much less of a problem if there is a PR that includes proper tests.

[jira-importer]

Henning Schmiedehausen commented

Good luck. I root for you.

[jira-importer]

Richard Eckart de Castilho commented

PS: PR was merged.

[jira-importer]

Richard Eckart de Castilho commented

Henning Schmiedehausen The next hurdle now is the release - which because primarily of the voting process may take a bit longer. The maintainer cannot simply push a button to do the release. A release must be prepared, a vote must be called for, at least three PMC members must cast a positive binding vote, then the release can be finalized. While this is legally an important process protecting open source and its developers under the umbrella of the ASF, it also is a tedious process. The more contributors a project can promote to PMC members, the faster the votes can be collected, the quicker a release can be done.

[jira-importer]

Slawomir Jaranowski commented

Elliotte Rusty Harold Henning Schmiedehausen

• PR - https://github.com/apache/maven-dependency-analyzer/pull/82 was merged to another component, we should have issue where change was dane

Now:

• issue is still open - I can't recognize what status is
• Henning Schmiedehausen is assigned but Elliotte Rusty Harold introduce change

How you want to manage it?

[jira-importer]

Richard Eckart de Castilho commented

The issue here can be closed once a the maven dependency plugin has upgraded to a new version of the dependency analyzer.

[jira-importer]

Henning Schmiedehausen commented

I unassigned myself. I have no further plans to work on this.