Gapic Python clients have high load time.

[s4shyam95]

I am switching the deprecated apitools clients with the new gapic clients and noticed that the gapic clients are much slower to load into memory than their deprecated counterpart.

Here is some comparison of load times taken by gapic clients :

For logging/v2 client:

• apitools : 0.4s
• GAPIC : 4.5s

For notebooks/v1 client:

• apitools : 0.1s
• GAPIC : 1.5s

With pre-compiled pyc files the load times improved, but not significantly. For logging/v2 client w/ pyc files:

• apitools : 0.1s
• GAPIC : 1.5s

For notebooks/v1 client w/ pyc files:

• apitools : 0.03s
• GAPIC : 0.4s

Unfortunately for our use-case, turnaround time is of the essence and the import time of GAPIC clients is still high.

Profiling the client import doesn't help much as the slowdown comes form importlib built-ins trying to load the client into memory.

I am aware of the option to add lazy loading like seen in the ads-template, but It requires a python version constraint that we are not ready for.

Would appreciate any other advice on improving the import speed of gapic clients.

[vam-google]

This is a known issue but unfortunately very difficult to fix. @s4shyam95 I believe this issue was created in the context of using gapics for gcloud. This integration is not happening anymore, so can you please clarify if it is still a relevant issue for you?

[s4shyam95]

I guess it's still relevant in the sense, that, not everything required should be loaded into memory, I see the gapic generator has moved to py3.9 (minimum?) or is only generate-time and not runtime?, In either case, is it possible to universally implement an ads-template like lazy loading solution so that only necessary artifacts are loaded into memory, as the profile shows most time goes into loading all the various types into memory by the __in_build__import calls.