Unable to disable telemetry

[ecatkins]

Describe the bug Unable to disable telemetry feature

To Reproduce

• Using os.environ['DEEPEVAL_TELEMETRY_OPT_OUT'] = "YES" at the top of my script
• Running deepeval test run test_xx.py over test cases

Expected behavior

• I would expect any telemetry behavior to be disabled. Instead I am still seeing attempted connections (which my system is blocking), see screenshot below

Screenshots

[penguine-ip]

hey @ecatkins, can you quickly print the env variable to see if it is indeed "YES" and resolves to true?

[penguine-ip]

@ecatkins I'm assuming this is no longer an issue?

[noa-weiss]

Was this ever fixed? I'm having the same problem.

[Jarzab3kDH]

+1 , when I set env variable to "YES" I get:

File "/home/vscode/.local/lib/python3.12/site-packages/deepeval/telemetry.py", line 235, in capture_metric_type posthog.capture(distinct_id=distinct_id, event=event, properties=properties) ^^^^^^^ NameError: name 'posthog' is not defined. Did you mean: 'Posthog'?

which is probably unrelated error, but the issue is that the this code gets executed, which imo shouldn't be the case.

Kind feedback - this is a "no-go" from my company perspective and destroys trust at very beginning.

The other thing is that you have stored API keys in git repository, which is another "no-go" for us as we cannot trust how you treat other sensitive information.

Again this is just kind feedback, as I spent couple of days testing it hoping I can use it in my project :-)

[penguine-ip]

@noa-weiss its fixed in the latest release, can you test it out and let us know? also you'll get a much better response time on our discord since the mesages there don't get missed as often

[satyaprakash1729]

@penguine-ip We're using 3.0.8. We're still seeing the issue.

[penguine-ip]

@satyaprakash1729 can you paste in the error message? Thanks

[satyaprakash1729]

Hi, looks like this is resolved in 3.1.0. I'll give it a try and update this thread. Thank you!

[mykolas-nice]

With 3.5.2 I have an issue.

I either get error uploading: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ when telemetry is enabled.

When I try to disable, I get "Error: name 'posthog' is not defined". The env is set to export DEEPEVAL_TELEMETRY_OPT_OUT=1 export DEEPEVAL_TELEMETRY_OPT_OUT=YES does not work.

Setting os.environ['DEEPEVAL_TELEMETRY_OPT_OUT'] = "YES" on the top of the script does not help either. I get the https error.

[penguine-ip]

Hey @mykolas-nice , from 3.5.0 onwards we've standardized all the env variables to boolean 0 or 1 flags, you can see the latest in our docs here: https://deepeval.com/docs/data-privacy

Let me know if this helps

[mykolas-nice]

Hey, @penguine-ip, thanks for the reply.

I've tried to set DEEPEVAL_TELEMETRY_OPT_OUT=1, and after this during metrics evaluation I get the error: Error: name 'posthog' is not defined

[mykolas-nice]

Tried version 3.4.0 - and found no issues. No errors appeared in simple execution, nor with DEEPEVAL_TELEMETRY_OPT_OUT set to YES or 1.