Fourier Feature Embedding#

class FourierFeatureEmbedding(input_dimension, output_dimension, sigma)[source]#

Bases: Module

Fourier Feature Embedding class for encoding input features using random Fourier features.This class applies a Fourier transformation to the input features, which can help in learning high-frequency variations in data. If multiple sigma are provided, the class supports multiscale feature embedding, creating embeddings for each scale specified by the sigma.

The FourierFeatureEmbedding augments the input by the following formula (3.10 of original paper):

\[\mathbf{x} \rightarrow \tilde{\mathbf{x}} = \left[ \cos\left( \mathbf{B} \mathbf{x} \right), \sin\left( \mathbf{B} \mathbf{x} \right)\right],\]

where \(\mathbf{B}_{ij} \sim \mathcal{N}(0, \sigma^2)\).

In case multiple sigma are passed, the resulting embeddings are concateneted:

\[\mathbf{x} \rightarrow \tilde{\mathbf{x}} = \left[ \cos\left( \mathbf{B}^1 \mathbf{x} \right), \sin\left( \mathbf{B}^1 \mathbf{x} \right), \cos\left( \mathbf{B}^2 \mathbf{x} \right), \sin\left( \mathbf{B}^3 \mathbf{x} \right), \dots, \cos\left( \mathbf{B}^M \mathbf{x} \right), \sin\left( \mathbf{B}^M \mathbf{x} \right)\right],\]

where \(\mathbf{B}^k_{ij} \sim \mathcal{N}(0, \sigma_k^2) \quad k \in (1, \dots, M)\).