Incorrect type referenced for `aspect.data` in google.cloud.dataplex_v1 client library documentation

[solomon-grant]

Determine this is the right repository

• [x] I determined this is the correct repository in which to report this bug.

Summary of the issue

Context The google-cloud-dataplex package depends on a protobuf wrapper package called proto-plus. Recent versions of the proto-plus dependency change the expected types of some google.cloud.dataplex_v1 objects.

In particular, when using proto-plus >= 1.24, the type of google.cloud.dataplex_v1.Aspect.data is proto.marshal.collections.maps.MapComposite rather than the documented google.protobuf.struct_pb2.Struct.

Expected Behavior:

The type of google.cloud.dataplex_v1.Aspect.data should be google.protobuf.struct_pb2.Struct. This works with proto-plus==1.23.0.

>>> from google.cloud import dataplex_v1
>>> client = dataplex_v1.CatalogServiceClient()
>>> entry = client.get_entry(name="my-entry-id")
>>> aspect = entry.aspects["my-aspect-id"]
>>> type(aspect.data)
<class 'google.protobuf.struct_pb2.Struct'>

Actual Behavior:

The type of google.cloud.dataplex_v1.Aspect.data is proto.marshal.collections.maps.MapComposite when using proto-plus>=1.24.0.

>>> from google.cloud import dataplex_v1
>>> client = dataplex_v1.CatalogServiceClient()
>>> entry = client.get_entry(name="my-entry-id")
>>> aspect = entry.aspects["my-aspect-id"]
>>> type(aspect.data)
<class 'proto.marshal.collections.maps.MapComposite'>

API client name and version

google-cloud-dataplex v2.6.0

Reproduction steps: code

file: main.py

from google.cloud import dataplex_v1

project_id = "my-google-cloud-project-id"
project_num = "my-google-cloud-project-number"
location = "my-google-cloud-region"

entry_group_id = "my-dataplex-catalog-entry-group-id"
aspect_type_id = "my-dataplex-catalog-aspect-type-id"
entry_id = "my-dataplex-entry-id"


def reproduce():
    client = dataplex_v1.CatalogServiceClient()
    entry_name = f"projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}"
    entry = client.get_entry(name=entry_name)
    aspect = entry.aspects[f"{project_num}.{location}.{aspect_type_id}"]
    print(type(aspect.data))


if __name__ == "__main__":
    reproduce()

Reproduction steps: supporting files

Reproduction steps: actual results

<class 'proto.marshal.collections.maps.MapComposite'>

Reproduction steps: expected results

<class 'google.protobuf.struct_pb2.Struct'>

OS & version + platform

No response

Python environment

Python 3.10.15

Python dependencies

google-cloud-dataplex 2.6.0 proto-plus 1.24.0

Additional context

This problem does not occur when using proto-plus v1.23.0.

[vchudnov-g]

Thanks for reporting this issue and providing the supporting data! We will investigate this and figure out how to best address it.

[ohmayr]

Thanks for reporting this issue. Taking a look!

[ohmayr]

@solomon-grant Here are my observations:

• With proto-plus==1.20.6 (which allows protobuf > 4.x.x), the type of google.cloud.dataplex_v1.Aspect.data is a Struct.
• With proto-plus > 1.20.6 (which implements a workaround for protobuf > 4.x.x), the type of google.cloud.dataplex_v1.Aspect.data is a MapComposite.
• With proto-plus < 1.20.6 (which allows protobuf < 4.x.x), the type of google.cloud.dataplex_v1.Aspect.data is a MapComposite.

I can confirm that the documentation incorrectly refers to the underlying protobuf types i.e. Struct for Aspect.data while the correct type is MapComposite. Protoplus is intended to wrap the underlying protobuf types with a more pythonic type.

The incorrect type (Struct) is only returned when using v1.20.6 of proto-plus.

I'm currently working on the next steps to mitigate this issue. Meanwhile, if you're still observing the return type to be a Struct while using any version of proto-plus other than 1.20.6 (without an incompatible protobuf version), kindly share more details about your environment and the installed dependencies and I can take a look into it!

[ohmayr]

Changing the type of this ticket from bug to process given that the issue no longer persists in the latest version of proto-plus. This issue is now a tracker for updating the documentation of google.cloud.dataplex_v1 client library documentation (and other affected libraries).