data sets / meta data to add?

[gaelforget]

Based on chat with L Talley :

• for ShipCruise type (CCHDO) we could also interface with GO-SHIP (https://usgoship.ucsd.edu) or GO-SHIP Easy Ocean (https://zenodo.org/records/13315689)
• for ShipCruise type (CCHDO) we need a list of valid expocode values (e.g., "33RR20160208"); there is table that can be found at https://usgoship.ucsd.edu/data/ but not clear how to download Hydrotable_Measurements.csv programmatically

[gaelforget]

example to read CCHDO's Hydrotable_Measurements.csv

df=CSV.read("Hydrotable_Measurements.csv",DataFrame)
cruises=unique(df.Cruise)
repeats_A16=findall(occursin.("A16",unique(df.Cruise)))

[gaelforget]

To get all summary files, this has worked :

url="https://cchdo.ucsd.edu/search?dtend=2025-01&download=woce%2csummary"
file=tempname()*".tar.gz"
Downloads.download(url,file)

[gaelforget]

update :

• OceanRobots.query , which uses CCHDO.extract_json_table under the hood, can be used to extract list of "expocode" (as of OceanRobots v0.2.13).
• seems easier and more general than using Hydrotable_Measurements.csv as suggested in https://github.com/JuliaOcean/OceanRobots.jl/issues/107#issuecomment-2385816285

code snippets :

using OceanRobots
url="https://cchdo.ucsd.edu/search?q=GO-SHIP"
CCHDO.extract_json_table(url)

using OceanRobots, CairoMakie
cruise=read(ShipCruise(),"33RR20160208")
plot(cruise,variable="salinity",colorrange=(33.5,35.0))

@DocOtak

[DocOtak]

Thanks for making me aware of this project! Some questions for my own understanding, with the caveat that CCHDO is mostly a python shop and as far as I know, none of us use Julia.

What are you doing with the summary files? They are somewhat a leftover from WOCE and can be very inconsistent in my opinion, we do generate some of them using the CF netCDF files as the source, but many of them are from the WOCE era.

What influenced the choice of which data format you are using? The CF netCDF files are meant to be the "modern" data file format, its internal structure is highly influenced by the argo _prof file, though all the variable names are different from Argo.

Are you using ERDDAP or something like to access data at all (not just CCHDO but maybe Argo)?

Is the search page on CCHDO used for other purposes than selecting which data to download?

I'll probably have more questions as we continue to discuss.

[gaelforget]

Thanks for the input and comments @DocOtak

What are you doing with the summary files? They are somewhat a leftover from WOCE and can be very inconsistent in my opinion, we do generate some of them using the CF netCDF files as the source, but many of them are from the WOCE era.

Nothing really at this time. Was looking for a short summary file in csv (useful to a new user like me, to avoid having to immediately learn about and dig into e.g. complex netcdf files) and found these su.txt were the closest thing.

What influenced the choice of which data format you are using?

Colleague who first walked me through the CCHDO site last spring. Sounded like they were most familiar with the WHP format.

The CF netCDF files are meant to be the "modern" data file format, its internal structure is highly influenced by the argo _prof file, though all the variable names are different from Argo.

Thanks for pointing this out. Sounds appealing.

Is this format documented? Found NCEI's in your submission docs, but am guessing it's not the same thing, is it?

Are you using ERDDAP or something like to access data at all (not just CCHDO but maybe Argo)?

Sometimes when available.

• gliders example uses http://spraydata.ucsd.edu/erddap/
• for NOAA buoys https://dods.ndbc.noaa.gov/thredds/
• for NOAA drifters ftp://ftp.aoml.noaa.gov/pub/phod/lumpkin/hourly/
• for Argo ftp://usgodae.org/pub/outgoing/argo , https://data-argo.ifremer.fr/dac , Dataverse , ...
• ...

Is there one for CCHDO? I looked at https://cchdo.ucsd.edu/data/ but does not seem to be it.

Is the search page on CCHDO used for other purposes than selecting which data to download?

Exploring and selecting data I guess. Seems the only searchable option, the json table output is nice, and search API is convenient.

At some point I was envisioning a Pluto notebook that would emulate the GUI part of e.g. https://cchdo.ucsd.edu/search?bbox=-75,-60,20,65 -- using your search API and , json table , and summary files such a notebook should not be hard to put together.

I'll probably have more questions as we continue to discuss.

Cool. Me too 🙂