opencensus-ext-requests usage with google-cloud-logging shows too many trace requests and slows down the application.

[sudharc]

Hi all, Background: I use Stackdriver to log messages and to store trace from my application. The application is hosted in GKE. The app uses Python 3.7 and the below libraries 'google-cloud-logging==1.15.0', 'google-cloud-trace==0.23.0', 'opencensus==0.7.10', 'opencensus-ext-stackdriver==0.7.2', 'opencensus-ext-requests==0.7.3', 'opencensus-ext-httplib==0.7.3'

I see traces of the below set of requests being traced multiple times and it is slowing down the application, as traces of all these requests are being uploaded to Stackdriver. As shown below, a simple api which just logs to Stackdriver and no other task itself is taking 9000+ms, which is very slow. The below set seem to be repeated multiple times.

Analysis: I guessed these traces came from URL requests happening within google-cloud-logging library and tried blacklisting by doing the below, but it did not help.

execution_context.set_opencensus_attr(
    'blacklist_hostnames',
    ['http://metadata.google.internal./computeMetadata/v1/', 'localhost', '127.0.0.1']
)
config_integration.trace_integrations(['requests'])

Request: I am reaching out for help from anyone to ignore tracing these requests i believe is happening within google-cloud-logging library. Thanks for any help in advance.

[lzchen]

Are you sending your requests via requests.get(...)? This could possibly be due to the requests extension wrapping the calls twice here and here. The requests.get logic runs through both of these. We should have a fix for this soon.

[sudharc]

@lzchen I am not explicitly sending requests in my code, but I believe there are requests.get() happening within google-cloud-logging library. These end up in trace and I m not able to ignore them from trace too.

[lzchen]

@sudharc Can you simply remove the trace_integration for requests if you do not want these to be tracked?

[bruce3557]

out team met similar problem also

[aabmass]

have you tried using excludelist_hostnames?

https://github.com/census-instrumentation/opencensus-python/blob/5ad2a7cc62fc9663d186e15636a7b0306c9a8d7a/contrib/opencensus-ext-requests/opencensus/ext/requests/trace.py#L76-L77