opentelemetry-python icon indicating copy to clipboard operation
opentelemetry-python copied to clipboard

AttributeError: '_Span' object has no attribute 'add_link'

Open mukund-ananthu opened this issue 1 year ago • 3 comments

Describe your environment

OS: (e.g, Ubuntu) Python version: Python 3.7 SDK version: (e.g., 1.25.0) API version: (e.g., 1.25.0)

What happened?

Python version: Python 3.7

LInk to the failed run: https://github.com/googleapis/python-pubsub/actions/runs/9637364916/job/26577285125?pr=1194 This error happens only for Python 3.7 and succeeds for other versions.

  1. tracer.start_as_current_span() is supposed to be returning a Span() object, on which add_link() method should work : https://github.com/open-telemetry/opentelemetry-python/blob/754fc36a408dd45e86d4a0f820f84e692f14b4c1/opentelemetry-sdk/src/opentelemetry/sdk/trace/init.py#L1086

Unexpectedly, the function returns a _Span() object when we use:

with tracer.start_as_current_span(name="foo", end_on_exit=False) as create_span1:

where create_span1 is of type _Span instead of Span().

Why is this happening?

self = <google.cloud.pubsub_v1.publisher._batch.thread.Batch object at 0x7f0e97085c90>

    def _commit(self) -> None:
        """Actually publish all of the messages on the active batch.
    
        This moves the batch out from being the active batch to an in progress
        batch on the publisher, and then the batch is discarded upon
        completion.
    
        .. note::
    
            This method blocks. The :meth:`commit` method is the non-blocking
            version, which calls this one.
        """
        with self._state_lock:
            if self._status in _CAN_COMMIT:
                self._status = base.BatchStatus.IN_PROGRESS
            else:
                # If, in the intervening period between when this method was
                # called and now, the batch started to be committed, or
                # completed a commit, then no-op at this point.
                _LOGGER.debug(
                    "Batch is already in progress or has been cancelled, "
                    "exiting commit"
                )
                return
    
        # Once in the IN_PROGRESS state, no other thread can publish additional
        # messages or initiate a commit (those operations become a no-op), thus
        # it is safe to release the state lock here. Releasing the lock avoids
        # blocking other threads in case api.publish() below takes a long time
        # to complete.
        # https://github.com/googleapis/google-cloud-python/issues/[80](https://github.com/googleapis/python-pubsub/actions/runs/9637364916/job/26577285125?pr=1194#step:5:81)36
    
        # Sanity check: If there are no messages, no-op.
        if not self._message_wrappers:
            _LOGGER.debug("No messages to publish, exiting commit")
            self._status = base.BatchStatus.SUCCESS
            return
    
        # Begin the request to publish these messages.
        # Log how long the underlying request takes.
        start = time.time()
    
        batch_transport_succeeded = True
        try:
            if self._client._open_telemetry_enabled:
                tracer = trace.get_tracer("com.google.cloud.pubsub.v1")
                links = []
                for wrapper in self._message_wrappers:
                    span = wrapper.create_span
                    if span.get_span_context().trace_flags.sampled:
                        links.append(trace.Link(span.get_span_context()))
                with tracer.start_as_current_span(
                    name=f"{self._topic} publish",
                    attributes={
                        "messaging.system": "com.google.cloud.pubsub.v1",
                        "messaging.destination.name": self._topic,
                        "gcp.project_id": self._topic.split("/")[1],
                        "messaging.batch.message_count": len(self._message_wrappers),
                        "messaging.operation": "publish",
                        "code.function": "_commit",
                    },
                    links=links if len(links) > 0 else None,
                    kind=trace.SpanKind.CLIENT,
                    end_on_exit=False,
                ) as publish_rpc_span:
                    ctx = publish_rpc_span.get_span_context()
                    for wrapper in self._message_wrappers:
                        if wrapper.create_span.get_span_context().trace_flags.sampled:
>                           wrapper.create_span.add_link(ctx)
E                           AttributeError: '_Span' object has no attribute 'add_link'

google/cloud/pubsub_v1/publisher/_batch/thread.py:301: AttributeError
------------------------------ Captured log setup ------------------------------
WARNING  opentelemetry.trace:__init__.py:521 Overriding of current TracerProvider is not allowed
- generated xml file: /home/runner/work/python-pubsub/python-pubsub/unit_3.7_sponge_log.xml -
=========================== short test summary info ============================
FAILED tests/unit/pubsub_v1/publisher/batch/test_thread.py::test_commit_otel_publish_rpc_span - AttributeError: '_Span' object has no attribute 'add_link'
1 failed, 1551 passed, 2 skipped in 24.65s
nox > Command py.test --quiet --junitxml=unit_3.7_sponge_log.xml --cov=google/cloud --cov=tests/unit --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=0 tests/unit failed with exit code 1
nox > Session unit-3.7 failed.

Steps to Reproduce

Already provided in the description above

Expected Result

Already provided in the description above

Actual Result

Already provided in the description above

Additional context

No response

Would you like to implement a fix?

None

mukund-ananthu avatar Jun 24 '24 00:06 mukund-ananthu

I noticed that the Python Open Telemetry docs indicate that it supports only Python 3.8 and above. Is that the reason for this issue: https://opentelemetry.io/docs/languages/python/#version-support

mukund-ananthu avatar Jun 24 '24 01:06 mukund-ananthu

@mukund-ananthu what version of the sdk are you using?

xrmx avatar Jun 24 '24 11:06 xrmx

xrmx opentelemetry-sdk=1.25.0

mukund-ananthu avatar Jun 24 '24 14:06 mukund-ananthu

@xrmx Could you PTAL the issue.

mukund-ananthu avatar Jul 06 '24 19:07 mukund-ananthu

@mukund-ananthu add_link is only available from sdk > 1.23.0, are you sure you're using sdk version 1.25.0 in the CI?

emdneto avatar Jul 06 '24 21:07 emdneto

If you check the setup.py file in my PR: https://github.com/googleapis/python-pubsub/pull/1194/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7 , you will see that the I just specify:

    "opentelemetry-api",
    "opentelemetry-sdk",

in the setup.py file. I assume this takes in the latest version of the library. When I check the pip freeze of my local development environment, I see that I am using 1.25.0

mukund-ananthu avatar Jul 06 '24 21:07 mukund-ananthu

The thing is, if you run pip install using python3.7 in CI, it will resolve to the last version that supports python3.7, which is 1.22.0. In 1.22.0, we didn't have add_link yet.

emdneto avatar Jul 06 '24 21:07 emdneto

Okay. What solution would you recommend in this case. To have code level checks for the python version and only enable / use OpenTelemetry if the python version is 3.8 or above

mukund-ananthu avatar Jul 06 '24 21:07 mukund-ananthu

@mukund-ananthu I think that depends on you. If you want to use features of a recent enough sdk you somewhat need to add a minimum version required and if that's not available on 3.7 I'll just make it a python 3.8+ feature.

xrmx avatar Jul 09 '24 08:07 xrmx

@mukund-ananthu Did you have a chance to take a look at the comment here? can we close this as done?

emdneto avatar Jul 15 '24 14:07 emdneto

Yes, I'm closing the bug. I see that the Open Telemetry Python docs also specify the min version of Python supported as 3.8. We will disable open telemetry for Python versions 3.7 or below. Thanks.

mukund-ananthu avatar Jul 15 '24 15:07 mukund-ananthu