PositionalEncoding

Defined in fynance.models.transformer

class PositionalEncoding(d_model, max_len=5000)

Bases: Module

Sinusoidal absolute positional encoding (Vaswani et al., 2017).

Adds a fixed position-dependent signal to the input embeddings so the attention layers can use the order of the sequence.

Parameters:

d_modelint

Embedding dimension.

max_lenint, optional

Maximum supported sequence length. Default 5000.

forward(x)

Add positional encoding to x of shape (B, T, d_model).