Orthogonal Block

class OrthogonalBlock(dim=-1, requires_grad=True)

Bases: Module

Orthogonal Block.

This block transforms an input tensor of shape \([N, M]\) into a tensor of the same shape whose columns are orthonormal. The block performs the Gram Schmidt orthogonalization, see here <https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process> for details.

Initialization of the OrthogonalBlock class.

Parameters:

• dim (int) – The dimension on which orthogonalization is performed. If -1, the orthogonalization is performed on the last dimension. Default is -1.

• requires_grad (bool) – If True, the gradients are computed during the backward pass. Default is True

forward(X)

Forward pass.

Parameters:

X (torch.Tensor) – The input tensor to orthogonalize.

Raises:

Warning – If the chosen dimension is greater than the other dimensions in the input.

Returns:

The orthonormal tensor.

Return type:

torch.Tensor

property dim

The dimension along which operations are performed.

Returns:

The current dimension value.

Return type:

int

property requires_grad

Indicates whether gradient computation is required for operations on the tensors.

Returns:

True if gradients are required, False otherwise.

Return type:

bool