[CDAP-19484] Revert CDAP-19448 and unpack artifacts in task workers again

[dli357]

This depends on the usage of Spark 3.2 which has fixed the bug of parsing a directory ending in .jar as a jar file. See https://github.com/scala/bug/issues/12019 for details.

[gitpod-io[bot]]

[dli357]

This depends on a netty version bump which depends on the removal of spark2 which is done in https://github.com/cdapio/cdap/pull/14797.

[tivv]

@dli357 could you please clarify why does it depend on spark2 work? I am asking because even though we remove spark2 from our code we still support it and we need to be sure users can still use their spark2 clusters (e.g. dataproc 1.5). Would this netty version bump still allow it?

[dli357]

This work depends on bumping the netty version since the PooledByteBufAllocator constructor used by this version of spark3 does not exist; unfortunately, bumping the netty version causes other compatibility issues with spark2 which cause tests to fail across the board.