[QUESTION] Can i apply to darts for multivariate regression problems

[philipp-leibner]

Hello,

I am currently working on a multivariate regression problem, where I need to predict for example 4 output features from 3 input features for each timestep (i.e. no forecasting into the future but predicting of other features at identical timesteps).
Is it possible to apply Darts timeseries / models to these kind of problems? I am currently using my own TCN implementation but using Darts would allow me to test different models much more quickly. But I could not find any example that is similar to my problem.

[dennisbader]

Hi @philipp-leibner, and thanks for raising this question.
This is only possible with our SKLearnModel and any of its subclasses (xgboost, lightgbm, catboost, ... models) when you set lags=None and lags_future_covariates=[0].

You can then pass the external features as future_covariates and the 3 target features as series to fit(), predict(), etc.

This notebook gives detailed examples on how to use these models.

[philipp-leibner]

Thanks, I'll take a look! Do you think it would be possible/useful addition to extend this to the PyTorch multivariate models?

[daidahao]

@philipp-leibner

Thanks, I'll take a look! Do you think it would be possible/useful addition to extend this to the PyTorch multivariate models?

One way to do this is to transform this regression problem into a pseudo-forecasting problem. To start, we would set equal input_chunk_length and output_chunk_length for a torch model, use the input features as past covariates and output features as future targets (labels), and zero out all past targets.

To implement this, you would create a custom TorchTrainingDataset and a TorchInferenceDataset which would return:

• zeros for past targets.
• 3 input features as past covariates.
• 4 output features as future targets.

Then, use TorchForecastingModel.fit_from_dataset() for training and .predict_from_dataset() for prediction.

Note that all torch models are originally forecasting models, so might have issues fitting the regression problem.

Here is a code snippet using TiDEModel:

import numpy as np

from darts import TimeSeries, concatenate
from darts.models import TiDEModel
from darts.utils.data import TorchInferenceDataset, TorchTrainingDataset
from darts.utils.timeseries_generation import linear_timeseries

INPUT_CHUNK_LENGTH = 10
NUM_TARGETS = 4
NUM_COVARIATES = 3

series = concatenate(
    [linear_timeseries(length=100, dtype=np.float32)] * (NUM_TARGETS + NUM_COVARIATES),
    axis=1,
)

model = TiDEModel(
    input_chunk_length=INPUT_CHUNK_LENGTH,
    output_chunk_length=INPUT_CHUNK_LENGTH,
    n_epochs=100,
)

class CustomTrainingDataset(TorchTrainingDataset):
    def __init__(self, series: TimeSeries, input_chunk_length: int):
        super().__init__()
        self.series = series

    def __len__(self):
        return len(self.series) - INPUT_CHUNK_LENGTH + 1

    def __getitem__(self, idx):
        values = self.series.values()
        future_target = values[idx:idx+INPUT_CHUNK_LENGTH, :NUM_TARGETS]
        past_target = np.zeros_like(future_target)
        past_covariates = values[idx:idx+INPUT_CHUNK_LENGTH, NUM_TARGETS:]
        return (past_target, past_covariates, None, None, None, None, future_target)

class CustomInferenceDataset(TorchInferenceDataset):
    def __init__(self, series: TimeSeries, input_chunk_length: int):
        super().__init__()
        self.series = series

    def __len__(self):
        return len(self.series) - INPUT_CHUNK_LENGTH + 1

    def __getitem__(self, idx):
        values = self.series.values()
        past_target = np.zeros((INPUT_CHUNK_LENGTH, NUM_TARGETS), dtype=np.float32)
        past_covariates = values[idx:idx+INPUT_CHUNK_LENGTH, NUM_TARGETS:]
        schema = self.series[list(self.series.columns[:NUM_TARGETS])].schema()
        pred_start = self.series.time_index[idx]
        return (past_target, past_covariates, None, None, None, None, schema, pred_start)

training_dataset = CustomTrainingDataset(series, INPUT_CHUNK_LENGTH)
model.fit_from_dataset(training_dataset)

inference_dataset = CustomInferenceDataset(series, INPUT_CHUNK_LENGTH)
preds = model.predict_from_dataset(n=INPUT_CHUNK_LENGTH, dataset=inference_dataset)

print(preds[0])
"""
              linear  linear_1  linear_2  linear_3
time                                              
2000-01-01 -0.000442  0.009123  0.002302 -0.009029
2000-01-02  0.011361  0.013060  0.011476  0.005001
2000-01-03  0.029747  0.024672  0.017779  0.015330
2000-01-04  0.031975  0.025115  0.031227  0.018739
2000-01-05  0.042686  0.050145  0.046958  0.037424
2000-01-06  0.047978  0.047979  0.051188  0.037541
2000-01-07  0.062206  0.068548  0.060583  0.054505
2000-01-08  0.062708  0.080094  0.062349  0.078343
2000-01-09  0.084926  0.086681  0.085031  0.076075
2000-01-10  0.085398  0.085808  0.090905  0.085413

shape: (10, 4, 1), freq: D, size: 160.00 B
"""