DeepEnsembleSolverInterface#

class DeepEnsembleSolverInterface(problem, models, optimizers=None, schedulers=None, weighting=None, use_lt=True, ensemble_dim=0)[source]#

Bases: MultiSolverInterface

A class for handling ensemble models in a multi-solver training framework. It allows for manual optimization, as well as the ability to train, validate, and test multiple models as part of an ensemble. The ensemble dimension can be customized to control how outputs are stacked.

By default, it is compatible with problems defined by AbstractProblem, and users can choose the problem type the solver is meant to address.

An ensemble model is constructed by combining multiple models that solve the same type of problem. Mathematically, this creates an implicit distribution \(p(\mathbf{u} \mid \mathbf{s})\) over the possible outputs \(\mathbf{u}\), given the original input \(\mathbf{s}\). The models \(\mathcal{M}_{i\in (1,\dots,r)}\) in the ensemble work collaboratively to capture different aspects of the data or task, with each model contributing a distinct prediction \(\mathbf{y}_{i}=\mathcal{M}_i(\mathbf{u} \mid \mathbf{s})\). By aggregating these predictions, the ensemble model can achieve greater robustness and accuracy compared to individual models, leveraging the diversity of the models to reduce overfitting and improve generalization. Furthemore, statistical metrics can be computed, e.g. the ensemble mean and variance:

\[\mathbf{\mu} = \frac{1}{N}\sum_{i=1}^r \mathbf{y}_{i}\]

\[\mathbf{\sigma^2} = \frac{1}{N}\sum_{i=1}^r (\mathbf{y}_{i} - \mathbf{\mu})^2\]

DeepEnsembleSolverInterface#

This Page