ResidualFeedForward#

class ResidualFeedForward(input_dimensions, output_dimensions, inner_size=20, n_layers=2, func=<class 'torch.nn.modules.activation.Tanh'>, bias=True, transformer_nets=None)[source]#

Bases: Module

Residual Feed Forward neural network model class.

The model is composed of a series of linear layers with a residual connection between themm as presented in the following:

See also

Original reference: Wang, S., Teng, Y., and Perdikaris, P. (2021). Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing 43.5 (2021): A3055-A3081. DOI: 10.1137/20M1318043

Initialization of the ResidualFeedForward class.

Parameters:

  • input_dimensions (int) – The number of input components. The expected tensor shape is \((*, d)\), where * represents any number of preceding dimensions (including none), and \(d\) corresponds to input_dimensions.

  • output_dimensions (int) – The number of output components . The expected tensor shape is \((*, d)\), where * represents any number of preceding dimensions (including none), and \(d\) corresponds to output_dimensions.

  • inner_size (int) – The number of neurons for each hidden layer. Default is 20.

  • n_layers (int) – The number of hidden layers. Default is 2.

  • func (torch.nn.Module | list[torch.nn.Module]) – The activation function. If a list is passed, it must have the same length as n_layers. If a single function is passed, it is used for all layers, except for the last one. Default is torch.nn.Tanh.

  • bias (bool) – If True bias is considered for the basis function neural network. Default is True.

  • transformer_nets (list[torch.nn.Module] | tuple[torch.nn.Module]) – The two torch.nn.Module acting as transformer network. The input dimension of both networks must be equal to input_dimensions, and the output dimension must be equal to inner_size. If None, two EnhancedLinear layers are used. Default is None.

Raises:

RuntimeError – If the number of layers and functions are inconsistent.

forward(x)[source]#

Forward pass for the ResidualFeedForward model.

Parameters:

x (torch.Tensor | LabelTensor) – The input tensor.

Returns:

The output tensor.

Return type:

torch.Tensor | LabelTensor