creosote icon indicating copy to clipboard operation
creosote copied to clipboard
verified publisher icon fredrikaverpil

Multiple namespace packages cause clashes

Open pineapple-pokopo opened this issue 1 year ago • 1 comments

What happened?

Following the fix for #274, I noticed another issue with namespace packages. When a project has more than one namespace package as dependency, it causes creosote to attribute package usage to the wrong package. The packages need to use the same namespace though.

Reproduction steps

I created a small repro package with the following layout (using PDM as package manager):

$ tree .   
.
├── pdm.lock
├── pyproject.toml
└── src
    └── main.py

2 directories, 3 files

where pyproject.toml contains:

[project]
name = "namespace-repr"
version = "0.0.1"
requires-python = "==3.12.*"
dependencies = [
    "google-cloud-storage==3.1.0",
    "google-cloud-bigquery==3.31.0",
]

[dependency-groups]
dev = [
    "creosote==4.0.2",
]

[tool.pdm]
distribution = false

and main.py:

from google.cloud import storage

client = storage.Client()

The output of creosote then is:

$ creosote          
Found dependencies in pyproject.toml: google-cloud-bigquery, google-cloud-storage
No unused dependencies found! ✨

while google-cloud-bigquery is unused. Note the incorrect associated_imports for this package in the output pasted below.

Relevant log output

Creosote version: 4.0.2
Command: creosote --verbose
Arguments: Config(verbose=True, format='default', paths=['src'], sections=['project.dependencies'], exclude_deps=[], deps_file='pyproject.toml', venvs=['<REDACTED>/.venv'], features=[])
Parsing src/main.py
Imports found in code:
- ImportInfo(module=['google', 'cloud'], name=['storage'], alias=None)
Parsing pyproject.toml for dependencies...
['project.dependencies']: ['google-cloud-storage==3.1.0', 'google-cloud-bigquery==3.31.0']
Detected PEP-621 toml section in pyproject.toml
Found dependencies in pyproject.toml: google-cloud-bigquery, google-cloud-storage
Gathering all top_level.txt files in venv <REDACTED>/.venv...
Found <REDACTED>/.venv/lib/python3.12/site-packages/webencodings-0.5.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/grpcio-1.71.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_cloud_core-2.4.3.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/defusedxml-0.7.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/requests-2.32.3.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_resumable_media-2.7.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/pandocfilters-1.5.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_cloud_bigquery-3.31.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/fastjsonschema-2.21.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/pyasn1-0.6.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_cloud_storage-3.1.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_crc32c-1.7.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/python_dateutil-2.9.0.post0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/mistune-3.1.3.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/charset_normalizer-3.4.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/certifi-2025.1.31.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_auth-2.38.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/tornado-6.4.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/creosote-4.0.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_api_core-2.24.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/pyasn1_modules-0.4.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/bleach-6.2.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/googleapis_common_protos-1.69.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/pip_requirements_parser-32.0.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/grpcio_status-1.71.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/cachetools-5.5.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/six-1.17.0.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/proto_plus-1.26.1.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/MarkupSafe-3.0.2.dist-info/top_level.txt
Found <REDACTED>/.venv/lib/python3.12/site-packages/bleach/_vendor/html5lib-1.1.dist-info/top_level.txt
Gathering all top_level.txt files in venv <REDACTED>/.venv...
Found <REDACTED>/.venv/lib/python3.12/site-packages/pygments-2.19.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/protobuf-5.29.4.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/idna-3.10.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/traitlets-5.14.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/webencodings-0.5.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/grpcio-1.71.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/tinycss2-1.4.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_cloud_core-2.4.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/defusedxml-0.7.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/requests-2.32.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/attrs-25.3.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/soupsieve-2.6.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/pyparsing-3.2.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/rpds_py-0.24.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/jupyter_client-8.6.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/beautifulsoup4-4.13.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_resumable_media-2.7.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/urllib3-2.3.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/dotty_dict-1.3.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/pandocfilters-1.5.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_cloud_bigquery-3.31.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/fastjsonschema-2.21.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/jsonschema-4.23.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/nbformat-5.10.4.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/pyasn1-0.6.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/jupyter_core-5.7.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/nbconvert-7.16.6.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/referencing-0.36.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_cloud_storage-3.1.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/loguru-0.7.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_crc32c-1.7.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/nbclient-0.10.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/jsonschema_specifications-2024.10.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/python_dateutil-2.9.0.post0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/jupyterlab_pygments-0.3.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/mistune-3.1.3.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/charset_normalizer-3.4.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/rsa-4.9.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/packaging-24.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/certifi-2025.1.31.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/platformdirs-4.3.7.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/jinja2-3.1.6.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_auth-2.38.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/tornado-6.4.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/creosote-4.0.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/google_api_core-2.24.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/pyasn1_modules-0.4.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/pyzmq-26.3.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/typing_extensions-4.13.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/bleach-6.2.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/googleapis_common_protos-1.69.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/pip_requirements_parser-32.0.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/grpcio_status-1.71.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/cachetools-5.5.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/six-1.17.0.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/proto_plus-1.26.1.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/MarkupSafe-3.0.2.dist-info/RECORD
Found <REDACTED>/.venv/lib/python3.12/site-packages/bleach/_vendor/html5lib-1.1.dist-info/RECORD
Attempting to find import names...
[google-cloud-bigquery] found import name(s) via top_level.txt: google ⭐️
[google-cloud-bigquery] found import name via RECORD: google ⭐️
[google-cloud-storage] found import name(s) via top_level.txt: google ⭐️
[google-cloud-storage] found import name via RECORD: google ⭐️
Dependencies with populated 'associated_import' attribute are used in code. End result of resolve:
- DependencyInfo(name='google-cloud-bigquery', top_level_import_names=['google'], record_import_names=['google'], canonicalized_dep_name='google_cloud_bigquery', associated_imports=[ImportInfo(module=['google', 'cloud'], name=['storage'], alias=None), ImportInfo(module=['google', 'cloud'], name=['storage'], alias=None)])
- DependencyInfo(name='google-cloud-storage', top_level_import_names=['google'], record_import_names=['google'], canonicalized_dep_name='google_cloud_storage', associated_imports=[ImportInfo(module=['google', 'cloud'], name=['storage'], alias=None), ImportInfo(module=['google', 'cloud'], name=['storage'], alias=None)])
No unused dependencies found! ✨

pineapple-pokopo avatar Apr 02 '25 12:04 pineapple-pokopo

This issue has been open for 30 days with no activity. Remove the stale label or add a comment or it will be closed in another 30 days.

github-actions[bot] avatar May 05 '25 07:05 github-actions[bot]