earth2studio icon indicating copy to clipboard operation
earth2studio copied to clipboard

🐛[BUG]: NCAR_ERA5 returns wrong coords and fails reading from cache

Open swbg opened this issue 7 months ago • 0 comments

Version

main

On which installation method(s) does this occur?

No response

Describe the issue

NCAR_ERA5 returns coords with dimensions "latitude" and "longitude" instead of "lat" and "lon". This is incompatible with existing model coords.

Furthermore, when caching and then trying to run again to read from the cache, I am getting a "Segmentation Fault" when using >1 workers and "malloc(): unsorted double linked list corrupted Aborted" when using max_workers=1. On a different system, I am getting the message below when running with >1 workers:

$uv run python3 get_data.py
Traceback (most recent call last):
  File "/.../.venv/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info
    file = self._cache[self._key]
           ~~~~~~~~~~~^^^^^^^^^^^

Following script (get_data.py):

from earth2studio.data import NCAR_ERA5
from datetime import datetime

if __name__ == "__main__":
    data = NCAR_ERA5(max_workers=2)
    variables = [
        "t2m",
        "u10m",
        "v10m",
        "sp",
        "tcwv",
        "t1000",
        "t850",
        "t500",
        "u1000",
        "u850",
        "u500",
        "v1000",
        "v850",
        "v500",
        "q1000",
        "q850",
        "q500",
        "z1000",
        "z850",
        "z500"
    ]
    time = [datetime(2024, 4, 1)]
    data(time, variables)

Following environment:

uv venv
source .venv/bin/activate
uv pip install "earth2studio[data] @ git+https://github.com/NVIDIA/earth2studio.git@73ef0df33069a617d7d8be044b3750e7b8ffa9e8"

swbg avatar May 27 '25 14:05 swbg