Regex support for scanConfigurations and skipConfigurations

[jeremylong]

To make selecting which configurations are being scanned more robust regular expressions should be added to scanConfigurations and skipConfigurations. For backward compatability we should leave the current scan/skip configuration options but I suggest we add:

configurations {
   scan: [list of configurations to scan]
   rxScan: [list of configurations to scan defined using regular expressions]
   skip: [list of configurations to skip]
   rxSkip: [list of configurations to skip defined using a regular expression]
}

The scanConfigurations can be removed from the documentation and a warning about a deprecated property can be issued (same for skipConfigurations). With the proposed changes the original skipConfigurations and scanConfigurations should be treated as a deprecated short cut to configurations { skip: []. scan: [] } respectively.

Lastly, the scan and skip configurations were mutually exclusive - I do not believe this is necessary with the addition of regular expressions. Skip should take precedence over scan.

[roikonen]

What are these configurations? I looked into documentation in http://jeremylong.github.io/DependencyCheck/dependency-check-gradle/configuration.html but could not find info about what those configurations mean. Can you somehow configure from which path(s) to scan with them?

I'm trying to use your OWASP Gradle plugin to scan my NodeJS dependencies but it can not find any dependencies and I can't find how to tell to the plugin where to find the dependencies.

[jeremylong]

@roikonen sorry for the extremely delayed response. Dependency-check-gradle uses the gradle dependency management system. There are many built in configurations (test, compile, testRuntime, somethingMadeUpByAnotherPlugin, etc.) each represent a collection of artifacts and their dependencies. The skip configurations allows one to explicitly tell the gradle plugin which set of dependencies to scan.

For Node.js my guess is gradle is not managing the dependencies and they are subsequently being managed by another system (likely npm). Take a look at the node security project or the CLI version of dependency-check when 2.0.0 is released (hopefully this weekend).

Other enhancements are in the works for the gradle (and Maven) plugin so that it can also scan specific directories for dependencies rather then just the dependencies managed by the build tool.

[nlassai]

@jeremylong Is there a way to skip the sub projects inside a project using skipConfigurations or something? We are using gradle plugin. Thanks.

[jeremylong]

@nlassai apply the plugin to the rootProject instead of allprojects or subprojects?

[jrodguitar]

@nlassai did you ever find a way to skip the sub projects?

[Thorbear]

Is this enhancement very far down in the backlog? Working with the kotlin-kapt plugin in addition to the Android plugin, the list of configurations to ignore is getting long:

kapt
kaptAndroidTestDebug
kaptRelease
kaptDebug
kaptTestDebug
kaptAndroidTest
kaptTest
kaptTestRelease

The ability to just say "Ignore all configurations starting with kapt" would be awesome.

[jrodguitar]

For a multiproject scenario, the fix for #99 worked good (have the ability to skip projects). However a fix for this issue will be what developers will use the most in my opinion. Please let @Thorbear and me @jrodguitar know.

[Vampire]

@Thorbear

dependencyCheck {
   allprojects {
      configurations.all {
         if ((it.name.startsWith('kapt')) && !(it.name in skipConfigurations)) {
            skipConfigurations << it.name
         }
      }
   }
}

[Vampire]

@nlassai & @jrodguitar though it is not documented, since 5.0.0 there is scanProjects and skipProjects introduced with #99.

[Thorbear]

@Vampire Awesome, with some minor edits, that can even add full regex support:

apply plugin: 'org.owasp.dependencycheck'
dependencyCheck {
    quickQueryTimestamp = false    // when set to false, it means use HTTP GET method to query timestamp. (default value is true)
    formats = ['HTML', 'XML']
    def skipConfigurationPatterns = [
            "_classStructurekapt.*",
            "_internal_aapt2_binary",
            "androidApis",
            "kotlinCompiler.*",
            "lintClassPath"
    ]
    allprojects {
        configurations.all { configuration ->
            if (configuration.name in skipConfigurations) {
                return
            }
            skipConfigurationPatterns.each { pattern ->
                if (configuration.name.matches(pattern)) {
                    skipConfigurations << configuration.name
                }
            }
        }
    }
}

[wrprice]

Is it necessary to make separate (e.g. scan and rxScan) properties? Could you determine an entry in the list is intended to be a regular expression by some convention (e.g. start with ^)? Arguably, one could use closures or even plain Java streams to select/build the scan/skip lists w/out an enhancement.

[jeremylong]

The solution to the problem is the above comment: https://github.com/dependency-check/dependency-check-gradle/issues/22#issuecomment-575568801

[Vampire]

The solution to the problem is the above comment

Not really, it is just a work-around. Anything that involves reaching into other projects models is discouraged bad practice and latest when isolated projects become a reality will probably be problematic. It would still be nice if you could simply configure a regex that is checked by the plugin in AbstractAnalyze#shouldBe* at execution time.