kg-microbe icon indicating copy to clipboard operation
kg-microbe copied to clipboard

Update Fermentation Explorer Database URL

Open sunitj opened this issue 4 months ago • 0 comments

In the file download.yaml, the URL for Fermentation Explorer no longer exists. This results in a 404 error when running poetry run kg download

Error message:

❯ poetry run kg download       
/<...>/envs/kgm/lib/python3.12/site-packages/eutils/__init__.py:4: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
ERROR:root:Failed to download: https://raw.githubusercontent.com/thackmann/FermentationExplorer/main/Shiny/FermentationExplorer/data/database.csv
Downloading files:  85%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                       | 23/27 [00:00<00:00, 100.00it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/<...>/envs/kgm/lib/python3.12/site-packages/click/core.py", line 1442, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/site-packages/click/core.py", line 1363, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/site-packages/click/core.py", line 1226, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/<...>/Github/sunitj/kg-microbe/kg_microbe/run.py", line 52, in download
    kg_download(*args, **kwargs)
  File "/<...>/Github/sunitj/kg-microbe/kg_microbe/download.py", line 22, in download
    download_from_yaml(
  File "/<...>/envs/kgm/lib/python3.12/site-packages/kghub_downloader/download_utils.py", line 95, in download_from_yaml
    raise e
  File "/<...>/envs/kgm/lib/python3.12/site-packages/kghub_downloader/download_utils.py", line 90, in download_from_yaml
    download_fn(item, outfile_path, snippet_only)
  File "/<...>/envs/kgm/lib/python3.12/site-packages/kghub_downloader/download.py", line 153, in http
    with urlopen(req) as response:  # noqa: S310
         ^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/urllib/request.py", line 215, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/urllib/request.py", line 521, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/urllib/request.py", line 630, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/urllib/request.py", line 559, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/<...>/envs/kgm/lib/python3.12/urllib/request.py", line 639, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

The following manual steps seem to resolve the issue:

  1. Download the file from here: https://raw.githubusercontent.com/thackmann/FermentationExplorer/main/Shiny/FermentationExplorer/data/database.zip
  2. Unzip database.zip
  3. Rename database.csv to fermentation_explorer.csv
  4. Move the fermentation_explorer.csv to /data/raw

sunitj avatar Jul 22 '25 19:07 sunitj