Fourier Feature Embedding#

class FourierFeatureEmbedding(input_dimension, output_dimension, sigma)[source]#

Bases: Module

Fourier Feature Embedding class to encode the input features using random Fourier features.

This class applies a Fourier transformation to the input features, which can help in learning high-frequency variations in data. The class supports multiscale feature embedding, creating embeddings for each scale specified by the sigma parameter.

The Fourier Feature Embedding augments the input features as follows (3.10 of original paper):

\[\mathbf{x} \rightarrow \tilde{\mathbf{x}} = \left[ \cos\left( \mathbf{B} \mathbf{x} \right), \sin\left( \mathbf{B} \mathbf{x} \right)\right],\]

where \(\mathbf{B}_{ij} \sim \mathcal{N}(0, \sigma^2)\).

If multiple sigma are passed, the resulting embeddings are concateneted:

\[\mathbf{x} \rightarrow \tilde{\mathbf{x}} = \left[ \cos\left( \mathbf{B}^1 \mathbf{x} \right), \sin\left( \mathbf{B}^1 \mathbf{x} \right), \cos\left( \mathbf{B}^2 \mathbf{x} \right), \sin\left( \mathbf{B}^3 \mathbf{x} \right), \dots, \cos\left( \mathbf{B}^M \mathbf{x} \right), \sin\left( \mathbf{B}^M \mathbf{x} \right)\right],\]

where \(\mathbf{B}^k_{ij} \sim \mathcal{N}(0, \sigma_k^2) \quad k \in (1, \dots, M)\).

Fourier Feature Embedding#

This Page