feat: need to add a way to override the SSL verify context through the clie…

[cmosguy]

We need a way to pass in the a context to the httpx client library so that we can manage the untreated certs.

The only way to do this is by following the recommendation here:

https://www.python-httpx.org/advanced/ssl/

All this fixes the issue to work around the error:

httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

[Pijukatel]

Maybe we should rather allow passing custom http client to the ApifyClientAsync, ApifyClient init methods? What do you think @janbuchar , @vdusek ?

[janbuchar]

Maybe we should rather allow passing custom http client to the ApifyClientAsync, ApifyClient init methods? What do you think @janbuchar , @vdusek ?

Maybe :shrug: However, we would have to change up the _BaseHTTPClient a bit so that we wouldn't need to pass both the sync and async version to it as that would be super unwieldy.

But, first of all... where did you get this error @cmosguy? I don't see a reason for api.apify.com to have an expired certificate. Can you show us a reproduction?

[cmosguy]

Maybe we should rather allow passing custom http client to the ApifyClientAsync, ApifyClient init methods? What do you think @janbuchar , @vdusek ?

Maybe 🤷 However, we would have to change up the _BaseHTTPClient a bit so that we wouldn't need to pass both the sync and async version to it as that would be super unwieldy.

But, first of all... where did you get this error @cmosguy? I don't see a reason for api.apify.com to have an expired certificate. Can you show us a reproduction?

Hey @janbuchar I am using my own laptop with MacOS and have no idea really how to resolve or get around this. It is not a laptop I setup, it was given to me. I do have Zscaler installed, but I am not sure if that matters. Any solution you can provide to the client so that we can inject new context to override the SSL would be great. Unless you have other ideas.

[Pijukatel]

Hello, @cmosguy we had internal discussion about this and agreed to allow passing a custom http client to ApifyClientAsync, ApifyClient init methods, which will allow you to pass your specifically configured client to solve your use case.

Do you feel comfortable with creating such PR?

The other possibility is that our team will eventually do it, but it will take more time as we have other work planned for the near future.