An LSTM (long short-term memory) is a recurrent neural-network layer that processes a sequence one element at a time and carries two related states forward: a cell state that stores information and a hidden state that exposes a regulated output. Three learned gates control what to retain, add, and reveal at each step.
How an LSTM processes a sequence
At step t, an LSTM combines the current input xₜ with the previous hidden state hₜ₋₁ to calculate gate values and candidate content. It then updates its cell state cₜ and produces a new hidden state hₜ. That pair is carried to the next step. In the standard formulation documented by PyTorch, the calculations are:
As an Amazon Associate I earn from qualifying purchases.
- iₜ = σ(Wᵢᵢxₜ + bᵢᵢ + Wₕᵢhₜ₋₁ + bₕᵢ) — input gate
- fₜ = σ(Wᵢfxₜ + bᵢf + Wₕfhₜ₋₁ + bₕf) — forget gate
- gₜ = tanh(Wᵢgxₜ + bᵢg + Wₕghₜ₋₁ + bₕg) — candidate cell content
- oₜ = σ(Wᵢoxₜ + bᵢo + Wₕohₜ₋₁ + bₕo) — output gate
- cₜ = fₜ ⊙ cₜ₋₁ + iₜ ⊙ gₜ
- hₜ = oₜ ⊙ tanh(cₜ)
Here σ is the sigmoid function, tanh is the hyperbolic tangent, and ⊙ means element-wise multiplication. The gate values are learned functions of the current input and previous hidden state. They scale vectors rather than acting as literal on/off switches.
Free tools Windows power users keep installed
One-click scans. No signup required.
What the three gates control
Forget gate: scale the previous memory
The forget gate fₜ scales the previous cell state cₜ₋₁. Values nearer zero reduce the corresponding components; values nearer one retain more of them.
#1 Best Overall
Input gate: admit candidate information
The input gate iₜ scales the candidate content gₜ. The product is added to the retained part of the old cell state, giving the update cₜ = fₜ ⊙ cₜ₋₁ + iₜ ⊙ gₜ.
Output gate: regulate what is exposed
The output gate oₜ scales tanh of the updated cell state to produce the hidden state hₜ. The hidden state is the step’s output representation; it is also passed forward as part of the inputs to the next step’s gate calculations.
Cell state and hidden state are related but distinct
The cell state is the internal memory carried through the sequence. The hidden state is a regulated output derived from that memory and exposed at the current step. A useful analogy is a running notebook: the forget gate scales what remains, the input gate scales a proposed addition, and the output gate scales what is shown to the next computation. This analogy describes learned vector operations, not a literal notebook or a set of discrete switches.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Why LSTMs were developed
In recurrent networks, error signals used in training can decay as they are propagated across many time steps, making it difficult to learn relationships over extended intervals. Sepp Hochreiter and Jürgen Schmidhuber introduced LSTM in their 1997 paper to help preserve error flow over long lags using a memory mechanism and multiplicative gates. Their abstract reports that, in their experimental setting, LSTM could bridge “minimal time lags in excess of 1000 discrete-time steps” (Hochreiter and Schmidhuber, 1997). That historical result is not a guarantee that an LSTM will learn any distant relationship, nor a general capacity benchmark for current tasks.
Rank #3
- Used Book in Good Condition
Where LSTMs are used
LSTMs are designed for ordered inputs, where the order and context of elements matter. PyTorch’s sequence-modeling tutorial demonstrates language modeling and part-of-speech tagging; TensorFlow’s tutorial discusses recurrent networks in the context of time-series forecasting. These examples show possible applications, not that LSTMs outperform other model types on them.
For a particular project, compare candidate models on the same task and data. Consider validation performance, sequence length and dependency structure, training and inference cost, available data, and whether future context is available when the model must make a prediction. A bidirectional LSTM uses context from both directions, so it is unsuitable when future inputs will not yet be available at prediction time.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Input shapes in PyTorch
Sequence, batch, and feature axes must match the layer’s expected layout. PyTorch’s torch.nn.LSTM API uses the final axis for input features. With the default batch_first=False, batched input has shape (L, N, H_in); with batch_first=True, it has shape (N, L, H_in). An unbatched input has shape (L, H_in). Here L is sequence length, N is batch size, and H_in is the number of input features. If initial hidden and cell states are omitted, they default to zeros.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →The API also supports multiple layers, bidirectional operation, and optional projections with proj_size > 0. These options affect output and state dimensions; check the API’s shape definitions when using them rather than assuming every output has the same width as the hidden state. In TensorFlow’s time-series tutorial, a Keras LSTM cell is wrapped in an RNN layer that manages sequence results and state. Consult the documentation for the library version in use when implementing a model.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
