ANN

Module for Artificial Neural Network (ANN) Prediction.

class ANN(layers, function, stop_training, loss=None, optimizer=<class 'torch.optim.adam.Adam'>, lr=0.001, l2_regularization=0, frequency_print=10, last_identity=True)[source]

Bases: Approximation

Feed-Forward Artifical Neural Network (ANN).

Parameters:

  • layers (list) – ordered list with the number of neurons of each hidden layer.

  • function (torch.nn.modules.activation) – activation function at each layer. A single activation function can be passed or a list of them of length equal to the number of hidden layers.

  • stop_training (list) – list with the maximum number of training iterations (int) and/or the desired tolerance on the training loss (float).

  • loss (torch.nn.Module) – loss definition (Mean Squared if not given).

  • optimizer (torch.optim) – the torch class implementing optimizer. Default value is Adam optimizer.

  • lr (float) – the learning rate. Default is 0.001.

  • l2_regularization (float) – the L2 regularization coefficient, it corresponds to the “weight_decay”. Default is 0 (no regularization).

  • frequency_print (int) – the frequency in terms of epochs of the print during the training of the network.

  • last_identity (boolean) – Flag to specify if the last activation function is the identity function. In the case the user provides the entire list of activation functions, this attribute is ignored. Default value is True.

Example:
>>> import ezyrb
>>> import numpy as np
>>> import torch.nn as nn
>>> x = np.random.uniform(-1, 1, size =(4, 2))
>>> y = np.array([np.sin(x[:, 0]), np.cos(x[:, 1]**3)]).T
>>> ann = ezyrb.ANN([10, 5], nn.Tanh(), [20000,1e-5])
>>> ann.fit(x, y)
>>> y_pred = ann.predict(x)
>>> print(y)
>>> print(y_pred)
>>> print(len(ann.loss_trend))
>>> print(ann.loss_trend[-1])

Initialize an Artificial Neural Network.

Parameters:

  • layers (list) – Ordered list with the number of neurons of each hidden layer.

  • function – Activation function(s) for each layer.

  • stop_training – Stopping criteria for training (iterations and/or tolerance).

  • loss – Loss function to use. Default is MSELoss.

  • optimizer – Optimizer class to use. Default is Adam.

  • lr (float) – Learning rate. Default is 0.001.

  • l2_regularization (float) – L2 regularization coefficient. Default is 0.

  • frequency_print (int) – Frequency of printing during training. Default is 10.

  • last_identity (bool) – Whether the last activation is identity. Default is True.

_abc_impl = <_abc._abc_data object>
_build_model(points, values)[source]

Build the torch neural network model.

Constructs a feed-forward neural network with the specified layers and activation functions.

Parameters:

  • points (numpy.ndarray) – The coordinates of the training points.

  • values (numpy.ndarray) – The training values at the points.

_convert_numpy_to_torch(array)[source]

Converting data type.

Parameters:

array (numpy.ndarray) – input array.

Returns:

the tensorial counter-part of the input array.

Return type:

torch.Tensor.

_convert_torch_to_numpy(tensor)[source]

Converting data type.

Parameters:

tensor (torch.Tensor) – input tensor.

Returns:

the vectorial counter-part of the input tensor.

Return type:

numpy.ndarray.

static _list_to_sequential(layers, functions)[source]
fit(points, values)[source]

Build the ANN given ‘points’ and ‘values’ and perform training.

Training procedure information:

  • optimizer: Adam’s method with default parameters (see, e.g., https://pytorch.org/docs/stable/optim.html);

  • loss: self.loss (if none, the Mean Squared Loss is set by default).

  • stopping criterion: the fulfillment of the requested tolerance on the training loss compatibly with the prescribed budget of training iterations (if type(self.stop_training) is list); if type(self.stop_training) is int or type(self.stop_training) is float, only the number of maximum iterations or the accuracy level on the training loss is considered as the stopping rule, respectively.

Parameters:

  • points (numpy.ndarray) – the coordinates of the given (training) points.

  • values (numpy.ndarray) – the (training) values in the points.

predict(new_point)[source]

Evaluate the ANN at given ‘new_points’.

Parameters:

new_points (array_like) – the coordinates of the given points.

Returns:

the predicted values via the ANN.

Return type:

numpy.ndarray