avro icon indicating copy to clipboard operation
avro copied to clipboard

AVRO-3985: [JAVA] adding default serializable pacakge to trusted package list

Open abhilakshyadobhal opened this issue 10 months ago • 4 comments

What is the purpose of the change

With this change, the default packages will always be going to be part of trusted packages.

What is the fix?

  • Static Block Enhancement:
    The static block has been updated to always include both user-defined packages (if any) and the default packages (java.lang,java.math,java.io,java.net,org.apache.avro.reflect), while removing duplicate entries.

  • Constructor Updates:
    Constructors in SpecificDatumReader now call initializeTrustedPackages(), ensuring that every instance is correctly populated with the trusted packages.

Verifying this change

This change is already covered by existing tests, such as (please describe tests).

Documentation

  • Feature Impact:
    This commit is a bug fix for the package initialization in SpecificDatumReader and does not introduce any new features.

  • Code Documentation:
    Code comments, especially in the initializeTrustedPackages() method, have been updated to clearly explain the logic behind the inclusion of both user-defined and default packages.

JIRA Ticket

  • JIRA Ticket: HADOOP-19315 , https://issues.apache.org/jira/browse/AVRO-3985

abhilakshyadobhal avatar Mar 08 '25 13:03 abhilakshyadobhal

Previously, if the org.apache.avro.SERIALIZABLE_PACKAGES system property was set as "*", then only that was added to SERIALIZABLE_PACKAGES and to trustedPackages, and trustAllPackages() saw that trustedPackages was a one-element list containing only "*" so it returned true.

Now with this PR, trustedPackages will contain the default trusted packages as well, so it won't be a one-element list, and trustAllPackages() will return false.

That's why I think this should not be merged as is.

Is it possible to add a test case for that regression? I'm not sure how to isolate system property changes from other tests running in parallel.

KalleOlaviNiemitalo avatar Mar 08 '25 14:03 KalleOlaviNiemitalo

HADOOP-19315 "Bump avro from 1.9.2 to 1.11.4" was already marked as fixed. If an Avro change is still needed then perhaps that should be a separate AVRO issue in Jira.

KalleOlaviNiemitalo avatar Mar 08 '25 14:03 KalleOlaviNiemitalo

https://github.com/apache/avro/pull/3330 fixes AVRO-3985 so that trustedPackages is initialized no matter which constructor of SpecificDatumReader is used. So if that pull request is merged first, then the only user-visible change remaining in this https://github.com/apache/avro/pull/3333 will be that the default serializable packages are always included. At that point, this PR should be retitled.

KalleOlaviNiemitalo avatar Mar 11 '25 12:03 KalleOlaviNiemitalo

This might be obsolete now with https://github.com/apache/avro/pull/3525

martin-g avatar Oct 23 '25 10:10 martin-g