Improve dependencies management

[pj892031]

Is your feature request related to a problem? Please describe.

This issue describes a couple of issues in dependency management that should be improved. The impact is:

• depends on the build there is no guarantee that a service will use a specific library (see in example multiple Java Servlet API implementation)
• other services that use common libraries have an issue with solving the upgrade of transitive libraries because API ML uses strict politics (it is required to exclude transitive dependencies and then use dependency in the external project)
• solving dependencies update (for example avoiding a security vulnerability requires using exclude and a new version as dependencies in a lot of places inside the repo)
• there is no one place to define upgrading a specific library
  • it complicates future development
  • it is very error-prone

Describe the solution you'd like

1. stop using strict policy within !! operator

A lot of versions are defined like

https://github.com/zowe/api-layer/blob/6b966b26cd988170896ae888b5f05a882a9e9c45/gradle/versions.gradle#L4

This version should be replaced with a different approach

2. define BOM

The aim is to define a new module which define version of dependencies to use, example:

plugins {
    id 'java-platform'
    id 'maven-publish'
}

dependencies {
 api(platform(defined_depency_to_use_across_modules));
 constraints {
        api("com.google.guava:guava:31.1-jre") {
            because 'older versions are vulnerable to CVE-2018-10237 and CVE-2020-8908'
        }
}

This module will be defined as a dependency for each module. It will define the right version to use and dependency upgrade in one place.

3. dependency plugin manager

Modules like apiml-common don't define any dependency manager plugin, but services do, ie. in metrics-service:

https://github.com/zowe/api-layer/blob/6b966b26cd988170896ae888b5f05a882a9e9c45/metrics-service/build.gradle#L22

It is the reason why the command ./gradlew apiml-common:dependencies shows just javax.servlet:javax.servlet-api:4.0.1 as a dependency, but ./gradlew metrics-service:dependencies shows also javax.servlet:servlet-api:*" (there is a conflict). The older version of servlet API is excluded in the common libraries, but different dependency management put them again on the classpath (metrics-services has both and it can cause an issue - it depends on the order of libraries).

The solution could be:

• remove the current dependency management
• replace with Gradle one
• use excluding from the whole project (or using BOM)

There could be a similar issue about a different dependency (it requires more deep analysis).

4. using api instead of implementation for internal module

It is better to use scope api for internal modules. It has a more accurate meaning for submodules.

5. common-service-core module contains ehCache

The dependency is defined as implementation, even on this level should be rather compileOnly. It is used just as a common library, the responsibility of using implementation is on modules including this module.

6. review required libraries in the whole project

All modules should be reviewed if:

• the dependency is required
• the scope of dependency on the specific level (for some modules just compileOnly could be enough, see bullet 5)
• check if on the classpath is just one version of each library

7. consider the replacement of old libraries

• some original Java libraries (ie. javax.activation-api could be replaced by Jakarta)
  • it is mainly about Eureka dependencies, but many of the libraries could be upgraded (a lot of original java JAR were "repackaged" by Jakarta)
• consider also upgrade of transitive libraries
  • upgrade of transitive libraries can solve static analysis warning (ie. like in https://github.com/Netflix/netflix-commons/pull/31)

Describe alternatives you've considered None.

Willingness to help I will if necessary.

Additional context The issue very probably doesn't describe all issue about dependency management, it just collects all known by me and could be extended during the work on it.

[balhar-jakub]

#2529 is linked to this one.