earthkit-data icon indicating copy to clipboard operation
earthkit-data copied to clipboard

Make the date parser more flexible

Open malmans2 opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe.

It would be nice to make the date parser more flexible. For example, it would be useful to submit requests of monthly data. The snippet below shows a possible implementation that uses the strings /to/ and /by/ as MARS.

cc: @EddyCMWF as I've noticed you just opened #237

Describe the solution you'd like

import pandas as pd
import pytest


def date_parser(fmt="%Y-%m-%d", **request):
    date = request.pop("date", None)
    if date is None:
        return request

    if isinstance(date, str):
        date = date.lower()
        for string in (" ", "to", "by"):
            date = date.replace(string, "")
        if "//" not in date:
            dates = pd.to_datetime(date.split("/"))
        else:
            date_split = iter(date.split("//"))
            start = next(date_split)
            end = next(date_split)
            freq = next(date_split, "1D")
            if freq.isdigit():
                freq += "D"
            dates = pd.date_range(start=start, end=end, freq=freq, inclusive="both")
    else:
        dates = pd.to_datetime(date)
    return request | {"date": dates.strftime(fmt).tolist()}


@pytest.mark.parametrize(
    "date,expected",
    (
        (
            "2000/to/2000-12/by/1MS",
            {
                "date": [
                    "2000-01-01",
                    "2000-02-01",
                    "2000-03-01",
                    "2000-04-01",
                    "2000-05-01",
                    "2000-06-01",
                    "2000-07-01",
                    "2000-08-01",
                    "2000-09-01",
                    "2000-10-01",
                    "2000-11-01",
                    "2000-12-01",
                ]
            },
        ),
        (
            ["2000-01", "2000-02"],
            {"date": ["2000-01-01", "2000-02-01"]},
        ),
        # Old features
        (
            "2000-01-01",
            {"date": ["2000-01-01"]},
        ),
        (
            "2000-01-01/2000-01-02",
            {"date": ["2000-01-01", "2000-01-02"]},
        ),
        (
            "2000-01-01/to/2000-01-02",
            {"date": ["2000-01-01", "2000-01-02"]},
        ),
        (
            "2000-01-01/to/2000-01-03/by/2",
            {"date": ["2000-01-01", "2000-01-03"]},
        ),
    ),
)
def test_parse_date(date, expected):
    assert date_parser(date=date) == expected

Describe alternatives you've considered

No response

Additional context

No response

Organisation

B-Open / CADS-EQC

malmans2 avatar Oct 17 '23 14:10 malmans2