[BUG] Float quantiles are compared using a set, which sometimes causes failures due to floating-point rounding.

[ChernovAndrey]

Hello Team,

First of all, thank you for your great work — the model looks really strong so far!

I’ve noticed that inference sometimes fails when I provide unrolled_quantiles as an input parameter. Here’s the error I encountered: Chronos2 model inference failed: Unrolled quantiles must be a subset of the model's quantiles. Found: unrolled_quantiles=[0.10000000149011612, 0.20000000298023224], model_quantiles=[0.01, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 0.99]

As input, I provided unrolled_quantiles = [0.1, 0.2]. They are a subset of the model quantiles, but I assume that due to floating-point rounding, the subset check fails.

pred_df = pipeline.predict_df(
    context_df,
    future_df=future_df,
    prediction_length=24,
    quantile_levels=[0.1, 0.5, 0.9, 0.935],
    unrolled_quantiles=[0.1, 0.2],
    id_column="id",
    timestamp_column="timestamp",
    target="target",
)

Looking into the code, I believe the issue comes from this line:

if not set(unrolled_quantiles).issubset(self.quantiles)

Comparing quantiles using NumPy with a small tolerance could likely resolve this issue.

Additionally, I have two questions about the quantile input parameters. I tried to understand them from the code, but got a bit lost, so I’d really appreciate your clarification: 1. quantile_levels parameter – When I provide a quantile level like 0.935, it works even though this value is not part of model_quantiles. Do you interpolate the output in this case? 2. unrolled_quantiles parameter – From what I understand, you perform an autoregressive process somewhat similar to beam search over these quantiles. How is the final output selected? Could you briefly elaborate on the algorithm? I’d also be interested in any statistics on how much using unrolled_quantiles increases computation cost and how significantly it improves accuracy. Are there any recommended default values for this parameter?

Thank you in advance for your help, and sorry for the long message!

Best regards, Andrei

[abdulfatir]

@ChernovAndrey For all intents and purposes you should not really use unrolled_quantiles ever (that's the reason we did not expose it in the docstring). What you need is only the quantile_levels argument. When quantile levels fall in between the model's quantiles, they are linearly interpolated. The unrolled_quantiles parameter is only intended and relevant for very long-horizon forecasting (e.g., if your prediction horizon is > 1024).

[abdulfatir]

Also, I wasn't able to reproduce the floating point issue. Could you share a minimal working example to reproduce the issue?

[ChernovAndrey]

Hey @abdulfatir, There are several ways to reproduce this issue. In my case, it occurs during deserialization of the input parameters. However, the simplest way to trigger the problem is to pass unrolled_quantiles with a different floating-point type. For example::

import pandas as pd  
from chronos import Chronos2Pipeline

def main():
    pipeline = Chronos2Pipeline.from_pretrained(
        "s3://autogluon/chronos-2",
        device_map="cpu",   # CPU on macOS is fine; avoids MPS quirks
    )

    # Load historical target values and past values of covariates
    context_df = pd.read_parquet(
        "https://autogluon.s3.amazonaws.com/datasets/timeseries/electricity_price/train.parquet"
    )

    # (Optional) Load future values of covariates
    test_df = pd.read_parquet(
        "https://autogluon.s3.amazonaws.com/datasets/timeseries/electricity_price/test.parquet"
    )
    future_df = test_df.drop(columns="target")

    import numpy as np
    unrolled_quantiles = np.array([0.1, 0.2], dtype=np.float32)
    # Generate predictions with covariates
    pred_df = pipeline.predict_df(
        context_df,
        future_df=future_df,
        prediction_length=24,
        quantile_levels=[0.1, 0.5, 0.9, 0.935],
        unrolled_quantiles=list(unrolled_quantiles),
        id_column="id",
        timestamp_column="timestamp",
        target="target"
    )

    print(pred_df)

and I get the following error: ValueError: Unrolled quantiles must be a subset of the model's quantiles. Found: unrolled_quantiles=[0.1, 0.2], model_quantiles=[0.01, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 0.99]

[abdulfatir]

Thanks for sharing the example. The issue you face is due to serialization and then deserialization of the the variable. Please pass it directly as unrolled_quantiles=[0.1, 0.2]. That said, like I mentioned previously, unrolled_quantiles is not relevant for the use case you're targeting and this is also true for most use cases. It only becomes relevant when prediction_length > 1024.