Remove security vulnerabilities if possible

[yruslan]

Background

SonaType has created this dependency security report: https://sbom.lift.sonatype.com/report/T1-118f0f57da8c6b3097cc-bb6bb3ca7a4e7-1687241554-048abf44d6b64eb2a99d21507e643b0b

Vulnerable libraries include:

• Kafka Client v2.5.1 - this is explicit dependency that we can change
• Snappy Java (1.1.7.3) - this is a transitive dependency. Maybe we can switch to the latest Spark for default builds and it can solve it.

Feature

Update project dependencies to remove security vulnerabilities while keeping Pramen compatible with Spark 2.4.3+

Make the default build for Spark 3.4.0 or later

Example

Kafka client can be made spark version dependent if Spark requires certain version of the Kafka client

[lokm01]

By default you should probably aim towards 3.3.1+ or 3.4.1+ (when available)

On Tue, 20 Jun 2023 at 08:20, Ruslan Yushchenko @.***> wrote:

Background

SonaType has created this dependency security report:

https://sbom.lift.sonatype.com/report/T1-118f0f57da8c6b3097cc-bb6bb3ca7a4e7-1687241554-048abf44d6b64eb2a99d21507e643b0b

Vulnerable libraries include:

• Kafka Client v2.5.1 - this is explicit dependency that we can change
• Snappy Java (1.1.7.3) - this is a transitive dependency. Maybe we can switch to the latest Spark for default builds and it can solve it.

Feature

Update project dependencies to remove security vulnerabilities while keeping Pramen compatible with Spark 2.4.3+ Example

Kafka client can be made spark version dependent if Spark requires certain version of the Kafka client

— Reply to this email directly, view it on GitHub https://github.com/AbsaOSS/pramen/issues/215, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU3YM4KHEZNMGE2AIPJDE3XME6KRANCNFSM6AAAAAAZMY5Y4Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[yruslan]

Thanks Jan! Definitely!