A recurrent neural network (RNN) is a neural network that processes a sequence by updating an internal state as each new input arrives. That state lets earlier context influence later steps, while the same learned transition is reused across the sequence.
What “recurrent” means
A feed-forward network ordinarily processes an input without carrying a state from one sequence step to the next. An RNN does: it reads an input, updates its hidden state, then uses that state when processing the next input. PyTorch summarizes the idea as “a network that maintains some kind of state” in its Sequence Models and Long Short-Term Memory Networks tutorial.
As an Amazon Associate I earn from qualifying purchases.
The state carries information forward, but it is not a perfect record of everything the network has seen. It is a learned representation that can let earlier context affect later processing.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesHow an RNN updates its state
A basic recurrent update is written as h_t = f_W(h_{t-1}, x_t). Here, x_t is the current input, h_{t-1} is the previous hidden state, h_t is the new state, and f_W is a learned transition function. The same transition parameters are applied at each time step, so the model can process sequences of different lengths without needing a separate transition for each position.
#1 Best Overall
For a simple vanilla RNN, Stanford’s CS231n notes show the update as h_t = tanh(W_hh h_{t-1} + W_xh x_t). An output can then be computed from the state. This equation describes a simple form, not every architecture called an RNN.
What sequence tasks RNNs can handle
RNNs can be arranged for different relationships between inputs and outputs. A model might consume a sequence and produce another sequence, or read an image representation and generate a word sequence as a caption. Language modeling, image captioning, and sequence-to-sequence tasks are examples discussed in Stanford CS231n materials and its Spring 2026 course schedule.
Vanilla RNN, LSTM, and GRU are not identical
“RNN” can refer broadly to recurrent architectures. A vanilla, or Elman, RNN is a simpler form with a hidden state updated at each step. LSTM and GRU are gated recurrent variants: their mechanisms regulate how information flows through the model. They belong to the recurrent family, but their state designs differ from the basic vanilla recurrence.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
In PyTorch’s documented RNN layer, each layer combines the current input and previous hidden state using learned weights and biases, then applies tanh by default or ReLU if configured. These are implementation details for that API, not requirements that define every RNN.
Rank #3
Why vanilla RNNs can struggle with long sequences
During training, gradients are propagated across time steps. In a vanilla RNN, those gradients may vanish or explode over many steps, making distant dependencies difficult to learn. LSTM’s cell-state mechanism can make long-distance information easier to preserve, but it does not guarantee that gradient problems disappear. Which recurrent variant is suitable depends on the task and the sequence arrangement; no variant is universally best based on these facts alone.
Quick Recap
Best Value
Rank #4
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
