OrthogonalBlock

class OrthogonalBlock(dim=-1, requires_grad=True)

Bases: Module

Module to make the input orthonormal. The module takes a tensor of size \([N, M]\) and returns a tensor of size \([N, M]\) where the columns are orthonormal. The block performs a Gram Schmidt orthogonalization process for the input, see here <https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process> for details.

Initialize the OrthogonalBlock module.

Parameters:

• dim (int) – The dimension where to orthogonalize.

• requires_grad (bool) – If autograd should record operations on the returned tensor, defaults to True.

forward(X)

Forward pass of the OrthogonalBlock module using a Gram-Schmidt algorithm.

Raises:

Warning – If the dimension is greater than the other dimensions.

Parameters:

X (torch.Tensor) – The input tensor to orthogonalize. The input must be of dimensions \([N, M]\).

Returns:

The orthonormal tensor.

property dim

Get the dimension along which operations are performed.

Returns:

The current dimension value.

Return type:

int

property requires_grad

Indicates whether gradient computation is required for operations on the tensors.

Returns:

True if gradients are required, False otherwise.

Return type:

bool