LongShortTermMemory¶

Defined in fynance.models.lstm

class LongShortTermMemory(X, y, drop=None, x_type=None, y_type=None, bias=True, forward_activation=nn.Softmax, hidden_activation=nn.Tanh, hidden_state_size=None, memory_activation=nn.Tanh, memory_state_size=None, forget_activation=nn.Sigmoid, update_activation=nn.Sigmoid, output_activation=nn.Sigmoid)[source]

Bases: _OutputLayerMixin, LSTMCell

Long Short-Term Memory neural network.

Full LSTM model: _LSTMCell four-gate architecture followed by a forward output projection. The cell state C and hidden state H are threaded through the sequence, allowing the model to carry information across many time steps without the vanishing-gradient pathology that limits RecurrentNeuralNetwork. Use it for sequence modeling tasks where dependencies span dozens of steps (intraday return series, multi-day momentum signals, regime detection).

Parameters:

X, yarray-like or int

If it’s an array-like, respectively inputs and outputs data.
If it’s an integer, respectively dimension of inputs and outputs.

dropfloat, optional

Probability of an element to be zeroed.

forward_activationtorch.nn.Module, optional

Activation functions, default is Softmax.

hidden_activation, memory_activationtorch.nn.Module, optional

Activation functions for respectively hidden and memory state, default both are Tanh function.

hidden_state_size, memory_state_sizeint, optional

Size of respectively hidden and memory states. Default hidden state is the same size as input; default memory state is the same size as hidden state.

forget_activation, update_activation, output_activation

torch.nn.Module, optional

Activation functions for respectively forget, update and output gate, default are Sigmoid function for all three.

Attributes:

criteriontorch.nn.modules.loss: A loss function.
optimizertorch.optim: An optimizer algorithm.
W_f, W_i, W_o, W_c, W_ytorch.nn.Linear: Respectively forget, update and output gate weights, weight to compute the candidate value for cell memory and forward weight.
f_f, f_i, f_o, f_c, f_ytorch.nn.Module: Respectively activation function for forget, update and output gate, activation function to compute the candidate value for cell memory and forward activation function.

See also

fynance.models.rnn.RecurrentNeuralNetwork
fynance.models.gru.GatedRecurrentUnit

forward(X, H, C)[source]

Forward method.

Parameters:

X, H, Ctorch.Tensor: Respectively input data, hidden state and memory state.

Returns:

torch.Tensor: Output data.
torch.Tensor: Hidden state.
torch.Tensor: Memory state.

load_model(path, load_optimizer=False)

Save the model with this weights and parameters.

Parameters:

pathstr or os.PathLike object: Path to load the model.
load_optimizerbool, optional: If True, then load also the optimizer.

predict(X, H, C)[source]

Predicts outputs of neural network model.

Parameters:

Xtorch.Tensor: Inputs to compute prediction.
Htorch.Tensor: States of the model.
Ctorch.Tensor: Cell memory of the model.

Returns:

torch.Tensor: Outputs prediction.
torch.Tensor: Updated states of the model.
torch.Tensor: Cell memory of the model.

save_model(path, save_optimizer=False)

Save the model with this weights and parameters.

Parameters:

pathstr or os.PathLike object: Path to save the model.
save_optimizerbool, optional: If True, then save also the optimizer.

set_data(X, y, x_type=None, y_type=None)

Set data inputs and outputs.

Coerces X and y to torch.Tensor and caches them as self.X / self.y. After the call the attributes self.T (number of observations), self.N (input columns) and self.M (output columns) are set.

Parameters:

X, yarray-like: Respectively input and output data. Accepted types: numpy.ndarray, torch.Tensor, pandas.DataFrame. Shapes must be (T, N) and (T, M) respectively.
x_type, y_typetorch.dtype, optional: Target dtypes for the resulting tensors. Default is None, which preserves the input dtype.

Returns:

BaseNeuralNet: self, to allow chaining.

Raises:

ValueError: If self.N / self.M were already set and X / y do not match, or if X and y have different lengths.

set_lr_scheduler(lr_scheduler, **kwargs)

Set dynamic learning rate.

Parameters:

lr_schedulertorch.optim.lr_scheduler._LRScheduler: Method from torch.optim.lr_scheduler to wrap self.optimizer, cf module torch.optim.lr_scheduler in PyTorch documentation [2].
**kwargs: Keyword arguments to pass to the learning rate scheduler.

References

[2]

https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate

set_optimizer(criterion, optimizer, params=None, **kwargs)

Set the optimizer object.

Set optimizer object with specified criterion as loss function and any kwargs as optional parameters.

Parameters:

criterionCallabletorch.nn.modules.loss: A loss function.
optimizertorch.optim.Optimizer: An optimizer algorithm.
paramsobject or iterable object: Layer of parameters to optimize or dicts defining parameter groups. If set to None then all parameters of model will be optimized. Default is None.
**kwargs: Keyword arguments of optimizer, cf PyTorch documentation [1].

Returns:

BaseNeuralNet: Self object model.

References

[1]

https://pytorch.org/docs/stable/optim.html

set_seed(seed_torch=None, seed_numpy=None)

Set seed for PyTorch and NumPy random number generator.

Parameters:

seed_torch, seed_numpybool or int, optional: If seed is an int \(0 < seed < 2^32\) set respectively PyTorch and NumPy seed with the number. Otherwise if is True then choose a random number, else doesn’t set seed.

train_on(X, y, H, C)[source]

Trains the neural network model.

Parameters:

X, y, H, Ctorch.Tensor: Respectively inputs, outputs, states and cell memory to train model.

Returns:

torch.nn.modules.loss: Loss outputs.
torch.Tensor: Updated states of the model.
torch.Tensor: Cell memory of the model.