Autoregressive Single Model Solver#

Module for the autoregressive single model solver class.

class AutoregressiveSingleModelSolver(problem, model, optimizer=None, scheduler=None, weighting=None, loss=None, use_lt=False, eps=0.0, reset_weights_at_epoch_start=True, kwargs=None)[source]

Bases: AutoregressiveMixin, SingleModelSolver

Single-model solver for autoregressive learning problems.

This solver learns the time evolution of dynamical systems using a single model. It is intended for problems defined by time-series data and accepts only TimeSeriesCondition.

Given a sequence of states \(\{\mathbf{u}_t\}_{t=0}^{T}\), the solver trains a model \(\mathcal{M}\) to predict the next state from the current one:

\[\hat{\mathbf{u}}_{t+1} = \mathcal{M}(\mathbf{u}_t).\]

The autoregressive training objective minimizes the discrepancy between the predicted states \(\hat{\mathbf{u}}_{t+1}\) and the target states \(\mathbf{u}_{t+1}\) over the sequence:

\[\mathcal{L}_{\mathrm{problem}} = \frac{1}{T} \sum_{t=0}^{T-1} \mathcal{L} \left( \mathbf{u}_{t+1} - \hat{\mathbf{u}}_{t+1} \right),\]

where \(\mathcal{L}\) is the selected loss function, typically the mean squared error.

The solver supports adaptive weighting of autoregressive steps through the eps parameter. During training, each autoregressive step can contribute differently to the total loss depending on its accumulated difficulty. Steps with larger running losses are assigned larger weights, so that the solver focuses more on parts of the rollout where prediction errors tend to accumulate. The parameter eps controls the strength of this effect: eps = 0 disables adaptive weighting, while larger values increase the influence of high-loss steps on the final training objective.

Initialization of the AutoregressiveSingleModelSolver class.

Parameters:

  • problem (BaseProblem) – The problem to be solved.

  • model (torch.nn.Module) – The model used by the solver.

  • optimizer (TorchOptimizer) – The optimizer used by the solver. If None, the torch.optim.Adam optimizer with a learning rate of 0.001 is used. Default is None.

  • scheduler (TorchScheduler) – The scheduler used by the solver. If None, the torch.optim.lr_scheduler.ConstantLR scheduler with a factor of 1.0 is used. Default is None.

  • weighting (BaseWeighting) – The weighting strategy used to combine condition losses. If None, no weighting is applied. Default is None.

  • loss – The loss function used to compute residual losses. If None, torch.nn.MSELoss is used. Default is None.

  • use_lt (bool) – If True, the solver uses LabelTensors as input. Default is False.

  • eps (float | int) – The hyperparameter controlling the influence of the cumulative loss on the adaptive weights. Higher values of eps will lead to more aggressive weighting of steps with higher cumulative loss. Default is 0.0.

  • reset_weights_at_epoch_start (bool) – Whether to reset the running average and step count at the start of each epoch. If True, the adaptive weights will be recalibrated at the beginning of each epoch based on the new training dynamics. Default is True.

  • kwargs (dict) – Additional keyword arguments for preprocessing and postprocessing steps.