Add tweedie on loss file

[janrth]

Adding the tweedie deviance in order to be used for model evaluation (if one decides to do so).

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[elephaint]

Thanks for your contribution! Two things quickly come to mind (reviewing on my phone)

1. Run utils.ipynb, but don't clear it. It's used for documentation in this repo.
2. The proposed loss should not depend on sklearn, as the latter isn't a depedency for utilsforecast and I don't think it needs to be a dependency for this loss.

[mergenthaler]

@marcopeix, Olivier is away next week. Could you please help @janrth so we can merge his PR?

[jan-voids]

I have a solution, just have to commit it. Then you guys can review.

[janrth]

@marcopeix If you have time, you can have a look at the new version. I removed the sklearn dependency.

[marcopeix]

Hello! There's a small issue with the docstring, so building the docs fail.

Can you change the docstring of tweedie_deviance to this:

@_base_docstring
def tweedie_deviance(
    df: DFType,
    models: List[str],
    power: float = 1.5,  
    id_col: str = "unique_id",
    target_col: str = "y",
) -> DFType:
    """
    Compute the Tweedie deviance loss for one or multiple models, grouped by an identifier.

    Each group's deviance is calculated using the mean_tweedie_deviance function, which
    measures the deviation between actual and predicted values under the Tweedie distribution.

    The `power` parameter defines the specific compound distribution:
      - 1: Poisson
      - (1, 2): Compound Poisson-Gamma
      - 2: Gamma
      - >2: Inverse Gaussian

    Additional Parameter
    ----------
    power : float, optional (default=1.5)
        Tweedie power parameter defining the distribution.
    """

Then, you can add a cell below with:

show_doc(tweedie_deviance, title_level=4)

This should fix the issue.

Make sure to run all the cells of losses.ipynb, and not clear the outputs. Thanks!

[janrth]

@marcopeix I think I did what you asked for.

[jan-voids]

@marcopeix I hope this way of writing the doc string would work. What do you say?

@_base_docstring
def tweedie_deviance(
    df: DFType,
    models: List[str],
    power: float = 1.5,
    id_col: str = "unique_id",
    target_col: str = "y",
) -> DFType:
    """Compute the group‐wise Tweedie deviance for one or more models.

    For each group in `id_col`, computes mean Tweedie deviance between the
    actuals (`target_col`) and each model’s predictions, using
    `mean_tweedie_deviance`.

    Args:
        df (DFType): DataFrame of true values and predictions.
        models (List[str]): Columns in `df` with model forecasts.
        power (float, optional): Tweedie power parameter:
            - 1: Poisson
            - (1,2): Compound Poisson–Gamma
            - 2: Gamma
            - >2: Inverse Gaussian  
            Defaults to 1.5.
        id_col (str, optional): Column to group by. Defaults to "unique_id".
        target_col (str, optional): Column of true values. Defaults to "y".

    Returns:
        DFType: DataFrame (same type as `df`) with one row per group and one
        column per model name, containing the average deviance.
    """

[marcopeix]

Hi @janrth , I found the last error that blocks this PR. In the mean_tweedie_deviance function, the dosctring should be:

def mean_tweedie_deviance(y_true: ArrayLike, y_pred: ArrayLike, power: float):
    """
    Compute the average Tweedie deviance between true values and predictions.

    The Tweedie deviance is defined differently depending on the power parameter:
      - power = 0: equivalent to mean squared error.
      - power = 1: equivalent to mean Poisson deviance.
      - power = 2: equivalent to mean gamma deviance.
      - other powers: general Tweedie deviance.

    Parameters
    ----------
    y_true : array-like
        Ground truth (correct) target values. Must be convertible to a NumPy array of floats.
    y_pred : array-like
        Predicted target values. Must be convertible to a NumPy array of floats and strictly positive.
    power : float
        Tweedie power parameter. Determines the distribution:
        0 for normal, 1 for Poisson, 2 for gamma, else general.
    
    Returns
    -------
    float
        Average Tweedie deviance over all samples
    """

Then, we should add a cell with:

show_doc(mean_tweedie_deviance, title_level=4)

From the changes, I see the output of the notebooks is still cleared, which should not be the case.

I made a separate PR to debug this issue (it was my first time encountering it) and I managed to have a PR passing all tests here (it's #168 ). We can either merge that one, or you can make the changes here.

Let me know what works best for you and sorry for this annoying bug.

[janrth]

Let me try it here. But I ran the nb and didn't clear it. So I wonder how that happened. Let me try again.

[janrth]

So it looks like the pre-commit hooks run the nb cleaning. I now committed with removing the nd dev cleaning. I hope that works now.

[marcopeix]

Hi @janrth , the linter didn't pass. You need to run nbdev_clean.

[janrth]

@marcopeix now I ran all hooks again. hope this works now :)

[elephaint]

Thanks, great work @janrth! Couple of things:

• The Polars loss doesn't return the same value for power=2 when y=0. Pandas will give an inf, Polars doesn't.
• In general we should try to avoid using .apply anywhere, Pandas UDF are very slow (we don't use them anywhere afaik in our libs). I'll try to see if I can make it a bit faster.
• There is no zero protection if power=2. I think this relates to the first point.

No need to work on it, I'll try to make those changes, will get back shortly.

[janrth]

@elephaint very true with the apply, I should have realised this! The issue with power=2 I didn't see at all. Thanks for taking care of it.

[elephaint]

@elephaint very true with the apply, I should have realised this! The issue with power=2 I didn't see at all. Thanks for taking care of it.

I refactored the code by using only Pandas/Polars expressions, added safeguards for powers>=2 and added more tests (all powers, NaNs and Infs).

Note that the comment re. apply also applies to Polars - map_elements is quite slow unfortunately on my machine. Below a timing test for 1000 series the previous code vs the refactor:

Power	Pandas - Expr	Pandas - apply	Speedup	Polars - Expr	Polars -map_elements	Speedup
0	0.0081	0.1766	22x	0.0028	3.989	1,425x
1	0.0162	0.1882	12x	0.0054	6.5637	1,216x
1.5	0.0221	0.1919	9x	0.0073	6.3635	872x
2	0.0173	0.1929	11x	0.0052	5.3419	1,027x
3	0.0223	0.2123	10x	0.0122	6.2615	513x
Average	0.0172	0.1924	11x	0.0066	5.7039	867x

I think it's good to go, wdyt @janrth?

[janrth]

That have been some big code changes you did. Thanks so much for your effort! @elephaint let's merge it :)