DeepEnsembleSupervisedSolver#

class DeepEnsembleSupervisedSolver(problem, models, loss=None, optimizers=None, schedulers=None, weighting=None, use_lt=False, ensemble_dim=0)[source]#

Bases: SupervisedSolverInterface, DeepEnsembleSolverInterface

Deep Ensemble Supervised Solver class. This class implements a Deep Ensemble Supervised Solver using user specified model``s to solve a specific ``problem.

An ensemble model is constructed by combining multiple models that solve the same type of problem. Mathematically, this creates an implicit distribution \(p(\mathbf{u} \mid \mathbf{s})\) over the possible outputs \(\mathbf{u}\), given the original input \(\mathbf{s}\). The models \(\mathcal{M}_{i\in (1,\dots,r)}\) in the ensemble work collaboratively to capture different aspects of the data or task, with each model contributing a distinct prediction \(\mathbf{y}_{i}=\mathcal{M}_i(\mathbf{u} \mid \mathbf{s})\). By aggregating these predictions, the ensemble model can achieve greater robustness and accuracy compared to individual models, leveraging the diversity of the models to reduce overfitting and improve generalization. Furthemore, statistical metrics can be computed, e.g. the ensemble mean and variance:

\[\mathbf{\mu} = \frac{1}{N}\sum_{i=1}^r \mathbf{y}_{i}\]

\[\mathbf{\sigma^2} = \frac{1}{N}\sum_{i=1}^r (\mathbf{y}_{i} - \mathbf{\mu})^2\]

During training the supervised loss is minimized by each ensemble model:

\[\mathcal{L}_{\rm{problem}} = \frac{1}{N}\sum_{i=1}^N \mathcal{L}(\mathbf{u}_i - \mathcal{M}_{j}(\mathbf{s}_i)), \quad j \in (1,\dots,N_{ensemble})\]

where \(\mathcal{L}\) is a specific loss function, typically the MSE:

\[\mathcal{L}(v) = \| v \|^2_2.\]

In this context, \(\mathbf{u}_i\) and \(\mathbf{s}_i\) indicates the will to approximate multiple (discretised) functions given multiple (discretised) input functions.

DeepEnsembleSupervisedSolver#

This Page