How to store the eddy tracking results

[chengyueLYH]

I would be so thankful if you could please help me. I have carried out eddy detection and realized eddy tracking through your sample code. How to store the eddy tracking results？？？l want to get save the path and boundary of each eddy.

[AntSimi]

Use to_netcdf method https://py-eddy-tracker.readthedocs.io/en/latest/_modules/py_eddy_tracker/observations/observation.html#EddiesObservations.to_netcdf

[chengyueLYH]

I use the data of aviso to detect the vortex and store it as NC data. It takes ten days. One day, the data is stored in an NC file. But when I track the vortex, the result is wrong. I don't know if there is something wrong with the function I called? Please provide some help，Another problem is how to store the test results of ten days in zarr. I will only store the test results of one day in zarr according to your instance code. I see that the sample data you provided is to store many test results in zarr, and the tracking code also reads the zarr data

import logging
from py_eddy_tracker.gui import GUI
from matplotlib import pyplot as plt
from netCDF4 import Dataset
import os
from py_eddy_tracker import start_logger
from py_eddy_tracker.data import get_remote_demo_sample
from py_eddy_tracker.featured_tracking.area_tracker import AreaTracker
import py_eddy_tracker.observations.observation
from py_eddy_tracker.tracking import Correspondances

py_eddy_tracker.observations.observation._display_check_warning = False

# file_objects = get_remote_demo_sample(
#     "eddies_med_adt_allsat_dt2018/Anticyclonic_2010_2011_2012"
# )[:20]
def get_local_demo_sample(folder_path):

    if not os.path.exists(folder_path):
        raise FileNotFoundError(f"The folder {folder_path} does not exist.")
    
    nc_files = []
    for filename in os.listdir(folder_path):
        if filename.endswith(".nc"):
            file_path = os.path.join(folder_path, filename)
            # nc_file = nc.Dataset(file_path, 'r')
            nc_files.append(file_path)
    
    return nc_files

def plot_eddy(ed):
    fig = plt.figure(figsize=(10, 5))
    ax = fig.add_axes([0.05, 0.03, 0.90, 0.94])
    ed.plot(ax, ref=-10, marker="x")
    lc = ed.display_color(ax, field=ed.time, ref=-10, intern=True)
    plt.colorbar(lc).set_label("Time in Julian days (from 1950/01/01)")
    ax.set_xlim(-180, 170), ax.set_ylim(-40, 40)
    ax.set_aspect("equal")
    ax.grid()
datapath = 'aviso_reult/Anticyclonic'
file_objects =get_local_demo_sample(datapath)

c = Correspondances(datasets=file_objects, class_method=AreaTracker, virtual=3)
c_first_run = Correspondances(
    datasets=file_objects, class_method=AreaTracker, virtual=10
)

start_logger().setLevel("INFO")
c_first_run.track()
start_logger().setLevel("WARNING")
with Dataset("correspondances.nc", "w") as h:
    c_first_run.to_netcdf(h)
# Next step are done only to build atlas and display it
c_first_run.prepare_merging()

# We have now an eddy object
eddies_area_tracker = c_first_run.merge(raw_data=False)
eddies_area_tracker.virtual[:] = eddies_area_tracker.time == 0
eddies_area_tracker.filled_by_interpolation(eddies_area_tracker.virtual == 1)
plot_eddy(eddies_area_tracker)

plt.show()

[chengyueLYH]

[AntSimi]

Zarr or netcdf is only a storage format and could be use to store identification or tracking. You need to apply a tracking on your identification file to create an atlas. You could also merge all identification result in one collection.

[thivinab]

Zarr or netcdf is only a storage format and could be use to store identification or tracking. You need to apply a tracking on your identification file to create an atlas. You could also merge all identification result in one collection.

Can you please tell me how to do this? I have identification files for each day as .nc files (separate for cyclonic and anticyclonic eddies). How should I proceed to merge these identifications for tracking?

[AntSimi]

Basics to do tracking are here: https://py-eddy-tracker.readthedocs.io/en/latest/run_tracking.html

[thivinab]

Basics to do tracking are here: https://py-eddy-tracker.readthedocs.io/en/latest/run_tracking.html

Thank you for your reply. I ran the tracking.yaml file (below);

PATHS:
  FILES_PATTERN: ./newCyclonic_*.nc
  SAVE_DIR: './output/'

VIRTUAL_LENGTH_MAX: 10  # Allow missing detections for up to 10 timesteps
TRACK_DURATION_MIN: 0  # Allow even the shortest trajectories

CLASS:
   MODULE: py_eddy_tracker.featured_tracking.area_tracker
   CLASS: AreaTracker

It created two output files; Cyclonic_correspondances.nc Cyclonic_untracked.nc

I tried to use the Cyclonic_correspondances.nc file with pet_track_anim.py (below);

import py_eddy_tracker_sample

from py_eddy_tracker.appli.gui import Anim
from py_eddy_tracker.observations.tracking import TrackEddiesObservations

data_path = "./Cyclonic_correspondances.nc"
a = TrackEddiesObservations.load_file(data_path)

eddy = a.extract_ids([80]).index(slice(0, 50))

a = Anim(eddy, sleep_event=1e-10, intern=True, figsize=(8, 3.5), cmap="viridis")
a.txt.set_position((17, 34.6))
a.ax.set_xlim(160, 220)
a.ax.set_ylim(-15, 12)
a.show(infinity_loop=False)

However I get the below error;

(eddenv) thivin_ab@THIVINAB:~/py-eddy-tracker/thivin/track/output$ python pet_track_anim.py 
Traceback (most recent call last):
  File "/home/thivin_ab/py-eddy-tracker/thivin/track/output/pet_track_anim.py", line 10, in <module>
    a = TrackEddiesObservations.load_file(data_path)
  File "/home/thivin_ab/mambaforge/envs/eddenv/lib/python3.10/site-packages/py_eddy_tracker/observations/observation.py", line 725, in load_file
    return cls.load_from_netcdf(filename, **kwargs)
  File "/home/thivin_ab/mambaforge/envs/eddenv/lib/python3.10/site-packages/py_eddy_tracker/observations/observation.py", line 903, in load_from_netcdf
    nb_obs = len(h_nc.dimensions[obs_dim])
KeyError: None

That obs_dim is missing.

Can you please guide me on how to correctly use the Cyclonic_correspondances.nc file for tracking? Thank you!

[AntSimi]

like listed in run_tracking page you must have 4 files in output of tracking, correspondances is an intern format to merge data, which are not usable lonely to do analyse

[thivinab]

like listed in run_tracking page you must have 4 files in output of tracking, correspondances is an intern format to merge data, which are not usable lonely to do analyse

Is it only possible to run an identification for a single day and create two files for each day? (https://py-eddy-tracker.readthedocs.io/en/latest/grid_identification.html) Is there a way to combine the identification files for multiple days as a single file?

[AntSimi]

There is a script to do that MergeEddies, but it's not a tracking. I am not sure to understand your question, which two files?

[thivinab]

There is a script to do that MergeEddies, but it's not a tracking. I am not sure to understand your question, which two files?

Thank you for your reply. I was referring to the cyclonic and anticyclonic identification files (as the two files).

I am running the tracking.yaml file (below);

PATHS:
  FILES_PATTERN: ./cyclonic_id_files/Cyclonic_*.nc
  SAVE_DIR: ./output/

VIRTUAL_LENGTH_MAX: 3
TRACK_DURATION_MIN: 3

However, only 2 files are created instead of 4.

I get the below error;

INFO 2025-01-16 15:20:13,700 tracking.prepare_merging :
                                        110 tracks identified
INFO 2025-01-16 15:20:13,700 tracking.prepare_merging :
                                        6710 observations will be join
        DEBUG 2025-01-16 15:20:13,700   tracking.       _copy :
                                                Copy done
Traceback (most recent call last):
  File "/home/thivin_ab/mambaforge/envs/eddenv/bin/EddyTracking", line 8, in <module>
    sys.exit(eddies_tracking())
  File "/home/thivin_ab/mambaforge/envs/eddenv/lib/python3.10/site-packages/py_eddy_tracker/appli/eddies.py", line 206, in eddies_tracking
    track(
  File "/home/thivin_ab/mambaforge/envs/eddenv/lib/python3.10/site-packages/py_eddy_tracker/appli/eddies.py", line 353, in track
    short_c.shorter_than(size_max=nb_obs_min)
  File "/home/thivin_ab/mambaforge/envs/eddenv/lib/python3.10/site-packages/py_eddy_tracker/tracking.py", line 616, in shorter_than
    translate = empty(i_keep_track.max() + 1, dtype="u4")
  File "/home/thivin_ab/mambaforge/envs/eddenv/lib/python3.10/site-packages/numpy/core/_methods.py", line 40, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity

[AntSimi]

how many file are select with ./cyclonic_id_files/Cyclonic_*.nc?