IMPR: 'import datajoint' slow on Mac amd64

[drewyangdev]

trafficstars

Improvement Request

Problem

Not really a problem, but Mac import datajoint is way less performant comparing to import datajoint on Linux.

Requirements

# make a directory
mkdir test_dj_import
cd ~/Downloads/test_dj_import

## make import.py
echo "import datajoint" > import.py

## CLI
conda create -n dj313 -c defaults python=3.13
pip install datajoint snakeviz

# generate cProfiler result
python -m cProfile -o result.log import.py
# filter and sort result to top 20 longest cumulative cpu time - check cProfile doc 
python -c 'import pstats; p=pstats.Stats("result.log");p.sort_stats("cumulative").print_stats(20)' > sorted_result.log

# if you want more
# visualize non filtered result.log, since can't visualize filtered result, the func recursive format was broken
pip install snakeviz
snakeviz result.log
# you will get an error saying too many depth, just change the filter to "depth: 3", "cutoff: 1/100"

Justification

On either Mac/amd64 or Linux/amd64, the results are in the same order, but it takes ~6 sec to import on Mac, but ~1 sec on Linux.

On Mac/amd64

# sort_result.log
cat sorted_result.log | grep .py
        1    0.000    0.000    6.138    6.138 import.py:1(<module>)
        1    0.000    0.000    6.136    6.136 /Users/yam/Documents/DataJoint/datajoint-python/datajoint/__init__.py:1(<module>)
        1    0.000    0.000    3.010    3.010 /Users/yam/Documents/DataJoint/datajoint-python/datajoint/diagram.py:1(<module>)
        1    0.000    0.000    2.531    2.531 /Users/yam/Documents/DataJoint/datajoint-python/datajoint/admin.py:1(<module>)
        1    0.000    0.000    1.986    1.986 /Users/yam/Documents/DataJoint/datajoint-python/datajoint/connection.py:1(<module>)
        1    0.000    0.000    1.543    1.543 /Users/yam/Documents/DataJoint/datajoint-python/datajoint/table.py:1(<module>)
        1    0.000    0.000    1.416    1.416 /Users/yam/opt/miniconda3/envs/dj313/lib/python3.13/site-packages/pandas/__init__.py:1(<module>)
        1    0.000    0.000    1.068    1.068 /Users/yam/opt/miniconda3/envs/dj313/lib/python3.13/site-packages/pandas/core/api.py:1(<module>)

On Linux/amd64

cat sorted_result.log | grep .py
        1    0.000    0.000    1.058    1.058 import.py:1(<module>)
        1    0.000    0.000    1.058    1.058 /home/drewyang/Documents/DataJoint/datajoint-python/datajoint/__init__.py:1(<module>)
        1    0.000    0.000    0.574    0.574 /home/drewyang/Documents/DataJoint/datajoint-python/datajoint/diagram.py:1(<module>)
        1    0.000    0.000    0.353    0.353 /home/drewyang/Documents/DataJoint/datajoint-python/datajoint/admin.py:1(<module>)
        1    0.000    0.000    0.310    0.310 /home/drewyang/Documents/DataJoint/datajoint-python/datajoint/connection.py:1(<module>)
        1    0.000    0.000    0.292    0.292 /home/drewyang/miniforge3/envs/dj313/lib/python3.13/site-packages/matplotlib/pyplot.py:1(<module>)
        1    0.000    0.000    0.203    0.203 /home/drewyang/Documents/DataJoint/datajoint-python/datajoint/table.py:1(<module>)
      106    0.002    0.000    0.185    0.002 /home/drewyang/miniforge3/envs/dj313/lib/python3.13/site-packages/matplotlib/artist.py:158(_update_set_signature_and_docstring)
      105    0.000    0.000    0.182    0.002 /home/drewyang/miniforge3/envs/dj313/lib/python3.13/site-packages/matplotlib/artist.py:125(__init_subclass__)
        1    0.000    0.000    0.169    0.169 /home/drewyang/miniforge3/envs/dj313/lib/python3.13/site-packages/pandas/__init__.py:1(<module>)

Please include steps to reproduce provided errors as follows:

• OS (MACOS | Linux)
• Python Version 3.13
• DataJoint Version 0.14.3

Screenshots

Snakeviz on Mac

Additional Research and Context

[github-actions[bot]]

This issue is stale because it has been open for 45 days with no activity.