How to build and train a custom model in darts?

[GuYith]

Is your feature request related to a current problem? Please describe. Is there any documentation related to implementing custom models in darts? I didn’t find it. I browsed some related issues. Inheriting TorchForecastingModel may be a way? I think if there are tutorials or APIs for implementing custom models, it will be helpful for enhancing the extension of Darts.

Describe proposed solution A clear and concise description of what the library should provide to solve missing functionality.

Describe potential alternatives A clear and concise description of any alternative solutions or existing features that might solve it.

Additional context Add any other context or screenshots about the feature request here.

[noahvand]

Recently, I tried to add a simple CNN model for time-series forecasting task based on Darts. And I cannot find any turtorial or guidance either. But I find some information from CHATGPT.

from darts import TimeSeries
from darts.models import TorchForecastingModel
from darts.utils.data import SequentialDataset
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import pandas as pd

class CustomCNN(nn.Module):
    def __init__(self, input_size):
        super(CustomCNN, self).__init__()
        self.conv1 = nn.Conv1d(1, 16, kernel_size=3, padding=1)
        self.act1 = nn.ReLU()
        self.fc1 = nn.Linear(input_size * 16, 50)
        self.act2 = nn.ReLU()
        self.fc2 = nn.Linear(50, 1)

    def forward(self, x):
        x = x.view(x.size(0), 1, -1)  # Reshape input for CNN
        x = self.conv1(x)
        x = self.act1(x)
        x = x.view(x.size(0), -1)  # Flatten
        x = self.fc1(x)
        x = self.act2(x)
        x = self.fc2(x)
        return x

class CNNForecastingModel(TorchForecastingModel):
    def __init__(self, input_chunk_length, output_chunk_length):
        self.model = CustomCNN(input_chunk_length)
        super().__init__(model=self.model, input_chunk_length=input_chunk_length, output_chunk_length=output_chunk_length)

    def train(self, training_series, val_series=None, epochs=15, batch_size=32):
        train_dataset = SequentialDataset(training_series, input_chunk_length=self.input_chunk_length, output_chunk_length=self.output_chunk_length)
        train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

        optimizer = torch.optim.Adam(self.model.parameters(), lr=0.001)
        loss_fn = nn.MSELoss()

        self.model.train()
        for epoch in range(epochs):
            for x_batch, y_batch in train_loader:
                optimizer.zero_grad()
                output = self.model(x_batch)
                loss = loss_fn(output, y_batch)
                loss.backward()
                optimizer.step()

# Example: Loading data from a CSV file
df = pd.read_csv('path_to_your_data.csv', parse_dates=['date_column'])
series = TimeSeries.from_dataframe(df, 'date_column', 'value_column')

# Split the series into train and validation sets
train, val = series.split_before(pd.Timestamp("YYYY-MM-DD"))

input_chunk_length = 30  # How many past time steps the model looks at
output_chunk_length = 1  # How many time steps ahead the model predicts

model = CNNForecastingModel(input_chunk_length, output_chunk_length)
model.train(train, val_series=val, epochs=10, batch_size=16)

predicted = model.predict(n=24)  # Predict the next 24 time steps
print(predicted)`

but not quite sure this will work :( can someone help me?

[madtoinou]

Hi @GuYith and @noahvand,

Adding a section on the documentation about how to implement a custom model is on the roadmap but not a high priority at the moment. But I'll try to give you the key elements for deep-learning models.

First, you need to determine if the model will support past, future and static covariates (check illustration here).

Based on this, you will be able to decide form which class it should inherit. At the moment, the options are : PastCovariatesTorchModel, FutureCovariatesTorchModel, DualCovariatesTorchModel, MixedCovariatesTorchModel, SplitCovariatesTorchModel and they are implemented here. Now, you can create your class with class NameModel([...]CovariatesTorchModel), which will requite the following methods : __init__() and _create_model() as well as a bunch of properties (check existing models).

Then, you can implement the actual neural network architecture in a separate class, class _NameModel(PL[...]CovariatesModule) which inherit for the Pytorch-Lightning module corresponding to the class selected above (implemented here). This class requires the __init__() and forward() methods to be defined.

That's pretty much it, for deterministic forecast. Probabilistic is a tad more complicated, and I would rather not into details here but I encourage you to check the implementation of any existing model as a reference for your work.

You might need to rely on the documentation for the dataset associated with the model parent class (here) to get the shape and order of the samples that the model will receive as well as the Pytorch-Lightning documentation to learn more about what is happening under the hood with the trainer (here).

[GuYith]

Hi @GuYith and @noahvand, 你好，、

Adding a section on the documentation about how to implement a custom model is on the roadmap but not a high priority at the moment. But I'll try to give you the key elements for deep-learning models.在文档中添加有关如何实现自定义模型的部分已列入路线图，但目前还不是最优先的事项。不过，我会尽力为您提供深度学习模型的关键要素。

First, you need to determine if the model will support past, future and static covariates (check illustration here).首先，您需要确定模型是否支持过去、未来和静态协变量（在此查看图例）。

Based on this, you will be able to decide form which class it should inherit. At the moment, the options are : PastCovariatesTorchModel, FutureCovariatesTorchModel, DualCovariatesTorchModel, MixedCovariatesTorchModel, SplitCovariatesTorchModel and they are implemented here. Now, you can create your class with class NameModel([...]CovariatesTorchModel), which will requite the following methods : __init__() and _create_model() as well as a bunch of properties (check existing models).在此基础上，您可以决定它应该继承哪个类。目前，可供选择的有 ： PastCovariatesTorchModel , FutureCovariatesTorchModel , DualCovariatesTorchModel , MixedCovariatesTorchModel , SplitCovariatesTorchModel 在此实现。现在，你可以用 class NameModel([...]CovariatesTorchModel) 创建你的类，这将需要以下方法： __init__() 和 _create_model() 以及一系列属性（查看现有模型）。

Then, you can implement the actual neural network architecture in a separate class, class _NameModel(PL[...]CovariatesModule) which inherit for the Pytorch-Lightning module corresponding to the class selected above (implemented here). This class requires the __init__() and forward() methods to be defined.然后，你可以在一个单独的类 class _NameModel(PL[...]CovariatesModule) 中实现实际的神经网络架构，该类继承了 Pytorch-Lightning 模块，与上面选择的类（在此实现）相对应。该类需要定义 __init__() 和 forward() 方法。

That's pretty much it, for deterministic forecast. Probabilistic is a tad more complicated, and I would rather not into details here but I encourage you to check the implementation of any existing model as a reference for your work.对于确定性预测来说，差不多就是这样。概率预测稍微复杂一些，我不想在此赘述，但我鼓励你们检查任何现有模型的实施情况，作为你们工作的参考。

You might need to rely on the documentation for the dataset associated with the model parent class (here) to get the shape and order of the samples that the model will receive as well as the Pytorch-Lightning documentation to learn more about what is happening under the hood with the trainer (here).您可能需要依靠与模型父类相关的数据集文档（此处）来获取模型将接收的样本的形状和顺序，以及 Pytorch-Lightning 文档（此处）来了解更多有关训练器的内部情况。

I'm glad to hear that. Thank you very much！