LongShortTermMemory¶

Defined in fynance.models.lstm

class LongShortTermMemory(X, y, drop=None, x_type=None, y_type=None, bias=True, forward_activation=nn.Identity, hidden_activation=nn.Tanh, hidden_state_size=None, memory_activation=nn.Tanh, memory_state_size=None, forget_activation=nn.Sigmoid, update_activation=nn.Sigmoid, output_activation=nn.Sigmoid)[source]

Bases: _OutputLayerMixin, LSTMCell

Long Short-Term Memory cell with output projection.

LSTM four-gate architecture (_LSTMCell) followed by a forward output projection. The caller supplies the hidden state H and cell state C; forward returns updated (Y, H, C). Like the other gated cells in this package, each of the ``T`` rows of ``X`` is processed independently — the cell does not loop over a time axis or thread H / C across rows on its own, so it is a stateless gated feed-forward cell. To model temporal dependencies the caller must thread H and C across successive steps explicitly. For built-in, causal sequence modeling prefer TemporalConvNet or Transformer.

Parameters:

X, yarray-like or int

If it’s an array-like, respectively inputs and outputs data.
If it’s an integer, respectively dimension of inputs and outputs.

dropfloat, optional

Probability of an element to be zeroed.

biasbool, optional

If True (default), the linear layers learn an additive bias.

forward_activationtorch.nn.Module, optional

Output activation, default is Identity (unconstrained regression output; pass nn.Softmax for a probability-simplex output).

hidden_activation, memory_activationtorch.nn.Module, optional

Activation functions for respectively hidden and memory state, default both are Tanh function.

hidden_state_size, memory_state_sizeint, optional

Size of respectively hidden and memory states. Default hidden state is the same size as input; default memory state is the same size as hidden state.

forget_activation, update_activation, output_activation

torch.nn.Module, optional

Activation functions for respectively forget, update and output gate, default are Sigmoid function for all three.

Attributes:

criteriontorch.nn.modules.loss: A loss function.
optimizertorch.optim: An optimizer algorithm.
W_f, W_i, W_o, W_c, W_ytorch.nn.Linear: Respectively forget, update and output gate weights, weight to compute the candidate value for cell memory and forward weight.
f_f, f_i, f_o, f_c, f_ytorch.nn.Module: Respectively activation function for forget, update and output gate, activation function to compute the candidate value for cell memory and forward activation function.

See also

fynance.models.rnn.RecurrentNeuralNetwork
fynance.models.gru.GatedRecurrentUnit

fit(X, y, epochs=1, x_type=None, y_type=None)[source]

Fit the model on (X, y) for epochs full-batch steps.

Conforms to the SignalModel contract. The hidden state H and cell state C are both zero-initialized once and threaded across epochs (detached between steps). An optimizer must have been registered with set_optimizer.

Parameters:

X, yarray-like: Input and output data (numpy / torch / polars), shapes (T, N) and (T, M).
epochsint: Number of full-batch training steps.
x_type, y_typetorch.dtype, optional: Target dtypes forwarded to set_data.

Returns:

LongShortTermMemory: self, to allow chaining.

forward(X, H, C)[source]

Forward method.

Parameters:

X, H, Ctorch.Tensor: Respectively input data, hidden state and memory state.

Returns:

torch.Tensor: Output data.
torch.Tensor: Hidden state.
torch.Tensor: Memory state.

load_model(path, load_optimizer=False)

Load the model weights and parameters from a file.

Parameters:

pathstr or os.PathLike object: Path to load the model.
load_optimizerbool, optional: If True, then load also the optimizer.

predict(X, H=None, C=None)[source]

Predicts outputs of neural network model.

Two calling conventions are supported:

predict(X) — conforms to the SignalModel contract: X may be array-like (coerced to a tensor), the hidden state and cell state are zero-initialized, and only the prediction tensor Y is returned.
predict(X, H, C) — explicit-state form: the updated states are threaded back, returning the (Y, H, C) tuple. C is zero-initialized when omitted.

In both cases X (and any supplied state) is moved to the model’s device.

Parameters:

Xarray-like or torch.Tensor: Inputs to compute prediction.
Htorch.Tensor, optional: States of the model. If None (default), a zero state is used and only the prediction is returned.
Ctorch.Tensor, optional: Cell memory of the model. Zero-initialized when None.

Returns:

torch.Tensor: Outputs prediction (when H is None).
tuple of torch.Tensor: (Y, H, C) outputs prediction and updated states (when H is provided).

save_model(path, save_optimizer=False)

Save the model with this weights and parameters.

Parameters:

pathstr or os.PathLike object: Path to save the model.
save_optimizerbool, optional: If True, then save also the optimizer.

set_data(X, y, x_type=None, y_type=None)

Set data inputs and outputs.

Coerces X and y to torch.Tensor and caches them as self.X / self.y. After the call the attributes self.T (number of observations), self.N (input columns) and self.M (output columns) are set.

Parameters:

X, yarray-like: Respectively input and output data. Accepted types: numpy.ndarray, torch.Tensor, polars.DataFrame. Shapes must be (T, N) and (T, M) respectively.
x_type, y_typetorch.dtype, optional: Target dtypes for the resulting tensors. Default is None, which casts floating-point inputs to torch.get_default_dtype() (float32 by default) and leaves integer inputs unchanged. See _set_data.

Returns:

BaseNeuralNet: self, to allow chaining.

Raises:

ValueError: If self.N / self.M were already set and X / y do not match, or if X and y have different lengths.

set_lr_scheduler(lr_scheduler, **kwargs)

Set dynamic learning rate.

Parameters:

lr_schedulertorch.optim.lr_scheduler._LRScheduler: Method from torch.optim.lr_scheduler to wrap self.optimizer, cf module torch.optim.lr_scheduler in PyTorch documentation [2].
**kwargs: Keyword arguments to pass to the learning rate scheduler.

References

[2]

https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate

set_optimizer(criterion, optimizer, params=None, **kwargs)

Set the optimizer object.

Set optimizer object with specified criterion as loss function and any kwargs as optional parameters.

Parameters:

criterionCallable, torch.nn.modules.loss: A loss function.
optimizertorch.optim.Optimizer: An optimizer algorithm.
paramsobject or iterable object: Layer of parameters to optimize or dicts defining parameter groups. If set to None then all parameters of model will be optimized. Default is None.
**kwargs: Keyword arguments of optimizer, cf PyTorch documentation [1].

Returns:

BaseNeuralNet: Self object model.

References

[1]

https://pytorch.org/docs/stable/optim.html

set_seed(seed_torch=None, seed_numpy=None)

Set seed for PyTorch and NumPy random number generator.

Each generator is only (re)seeded when its argument is provided: passing seed_torch alone leaves the global NumPy RNG untouched, and vice versa.

Parameters:

seed_torch, seed_numpybool or int, optional: If an int \(0 \leq seed < 2^{32}\), seed respectively the PyTorch and NumPy generator with that number. If True, draw a random seed. If None (default), leave that generator untouched.

Examples

>>> from fynance.models.mlp import MultiLayerPerceptron
>>> model = MultiLayerPerceptron(3, 1, layers=[4])
>>> model.set_seed(seed_torch=42)
>>> model.seed_torch
42
>>> model.seed_numpy is None
True

train_on(X, y, H, C)[source]

Trains the neural network model.

Parameters:

X, y, H, Ctorch.Tensor: Respectively inputs, outputs, states and cell memory to train model.

Returns:

torch.nn.modules.loss: Loss outputs.
torch.Tensor: Updated states of the model.
torch.Tensor: Cell memory of the model.