cylc
tutorial workflow: move away from datapoint
Datapoint is past its sell by and no longer supported. It has become increasingly unstable for our usage so we will need to look at moving away.
Its replacement is the "Weather Observations Website" or WOW - https://www.wow.metoffice.gov.uk/
Unfortunately I don't think we will be able to bundle API keys for WOW (in order to skip the registration step) so would require users to sign up.
The current tutorial extracts cloud cover from radar images which may or may not be available in WOW, if not we could use the "current weather" field. This would help us drop the Pillow dependency.
Note: Ideally the tutorial workflow would not have any additional dependencies beyond the regular cylc-flow dependencies.
Met Office DataPoint will be retired on 01 December 2025.
We now have a deadline for this.
Quick and dirty solution is just to remove live data altogether and rely on canned data. However, this might be a tad disappointing for users and does make the get_observations tasks a tad redundant (though the tutorial still requires them).
Current closest replacements:
Wind is currently being collected from Synops by default. The Rainfall Radar is the product we don't have an alternative to.
It looks like DataHub could do the job:
- One API key per endpoint, perfectly manageable.
- Windspeed data.
- Precipitation map.
The only issue is that it only seems to provide forecast data, not observation data.
Possible solution: AWS Open Data
https://registry.opendata.aws/met-office-uk-radar-observations/
- Requires AWS CLI:
aws s3 cp --no-sign-request s3://met-office-radar-obs-data/radar/2025/09/08/202509081200_ODIM_ng_radar_rainrate_composite_1km_UK.h5 ., or presumably Python equivelents? - Requires hdf3 library:
fh = h5py.File(data.hdf5); fh['dataset1']['data1']['data']
It's horrid, but would work - I'm talking to radar people about better solutions - I was hoping for example that we might be able to use data shared to then by the WMO.