snowflake-connector-python
snowflake-connector-python copied to clipboard
Make file_transfer_agent and pkg_resources imports lazy
Fixes #1253
I noticed that two modules take up a significant portion of import time but aren't necessary for many use cases. For instance pkg_resources
doesn't have to be imported if pandas isn't used. Similarly file_transfer_agent
isn't going to be used if only executing get requests.
Moving these imports from the top level to the function they are used in can improve import time from:
$ hyperfine 'python -c "import snowflake.connector"'
Benchmark 1: python -c "import snowflake.connector"
Time (mean ± σ): 466.0 ms ± 56.2 ms [User: 434.9 ms, System: 29.1 ms]
Range (min … max): 427.6 ms … 616.1 ms 10 runs
to:
$ hyperfine 'python -c "import snowflake.connector"'
Benchmark 1: python -c "import snowflake.connector"
Time (mean ± σ): 255.3 ms ± 4.9 ms [User: 233.8 ms, System: 21.8 ms]
Range (min … max): 248.9 ms … 264.9 ms 11 runs
This is of course the best case. If pandas is used pkg_resources
will get imported and no time will be saved there and if file transfer is used the time to import file transfer module will be paid on the first usage instead of the import time.
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅
I have read the CLA Document and I hereby sign the CLA
recheck
Closing this PR. Revived this with tests here: https://github.com/snowflakedb/snowflake-connector-python/pull/1512