Installation failing on Python 3.12 due to removal of imp package

[ethanwhite]

conda create -n retriever-test python=3
conda activate retriever-test
pip install retriever
retriever ls

Traceback (most recent call last):
  File "/home/ethan/miniconda3/envs/retriever-test/bin/retriever", line 5, in <module>
    from retriever.__main__ import main
  File "/home/ethan/miniconda3/envs/retriever-test/lib/python3.12/site-packages/retriever/__init__.py", line 7, in <module>
    from retriever.lib.engine_tools import set_proxy, create_home_dir
  File "/home/ethan/miniconda3/envs/retriever-test/lib/python3.12/site-packages/retriever/lib/__init__.py", line 4, in <module>
    from .datasets import datasets
  File "/home/ethan/miniconda3/envs/retriever-test/lib/python3.12/site-packages/retriever/lib/datasets.py", line 1, in <module>
    from retriever.lib.scripts import SCRIPT_LIST, get_script, get_dataset_names_upstream
  File "/home/ethan/miniconda3/envs/retriever-test/lib/python3.12/site-packages/retriever/lib/scripts.py", line 2, in <module>
    import imp
ModuleNotFoundError: No module named 'imp'

Here's the general solution https://stackoverflow.com/a/52794560

Note that this package has been deprecated since at least 2018, so we should probably develop a process for paying closer attention to deprecation warnings and solving them before they eventually break things.

[dikwickley]

this comment seems helpful https://github.com/DataDog/dd-trace-py/issues/1819#issue-746576174

[arnavk23]

• Solution: Use an Older Python Version Since retriever hasn’t been updated for Python 3.12 compatibility, downgrade your Python environment to 3.11:

conda create -n retriever-test python=3.11 conda activate retriever-test pip install retriever retriever ls

• Another Solution : If you must use Python 3.12, you can modify the package to use importlib instead of imp.
  ~/.conda/envs/retriever-test/lib/python3.12/site-packages/retriever/lib/scripts.py import importlib conda deactivate # Restart the environment conda activate retriever-test retriever ls