Recommended Free Tools
A gated recurrent unit (GRU) is a recurrent neural-network unit that processes a sequence one step at a time, carrying a hidden state forward as it goes. Two learned, elementwise gates regulate that state: the reset gate controls how much of the prior state contributes to a candidate, and the update gate controls how that candidate is blended with the old state.
What a GRU network does
At time step t, a GRU receives the current input xt and the previous hidden state ht−1. It uses them to calculate a new hidden state ht, which can carry information forward to later steps. In a language model, for example, the input might represent the current token while the hidden state summarizes information from earlier tokens.
As an Amazon Associate I earn from qualifying purchases.
The reset and update gates are learned controls, not hand-written rules or binary switches. A sigmoid activation gives each gate coordinate a value between zero and one, allowing different dimensions of the state to be regulated independently.
How the GRU gates update the hidden state
Reset gate: shaping the candidate
The reset gate rt controls how much of the previous hidden state contributes when the unit computes a candidate state nt. Lower values reduce that prior-state contribution for the relevant coordinates; higher values let more of it through.
#1 Best Overall
Update gate: blending old and candidate information
The update gate zt determines the balance between the previous state and the candidate state. In the convention used by PyTorch and Dive into Deep Learning, a value near one retains more of the old state, while a value near zero moves the new state toward the candidate.
PyTorch documents the following equations, where σ is the sigmoid function, tanh is the hyperbolic tangent, and ⊙ denotes elementwise multiplication:
Rank #2
rt = σ(Wirxt + bir + Whrht−1 + bhr)
zt = σ(Wizxt + biz + Whzht−1 + bhz)
nt = tanh(Winxt + bin + rt ⊙ (Whnht−1 + bhn))
ht = (1 − zt) ⊙ nt + zt ⊙ ht−1
PyTorch’s GRU reference documents this convention. The equations show why the gates should be understood as soft, coordinate-by-coordinate controls: their values determine proportions of contributions rather than switching an entire state on or off.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Why GRU equations can differ between frameworks
Frameworks do not necessarily place the reset multiplication at the same point in the candidate calculation. In PyTorch’s documented equation, the reset gate multiplies the recurrent transformation of the previous hidden state. The original formulation applies the reset gate to the prior hidden state before that transformation. PyTorch describes its placement as an efficiency choice.
Rank #3
This distinction matters when comparing implementations or transferring weights: check the framework’s definition rather than assuming that matching gate names guarantee identical equations.
Where GRUs came from and how they are used
Cho and colleagues introduced the reset-and-update hidden unit in their 2014 paper, “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.” Their encoder-decoder used recurrent networks to map a variable-length source sequence to a representation and generate or score a target sequence, with phrase scoring in statistical machine translation as the reported application. The authors wrote: “The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence.”
GRUs also appear in educational examples of sequence encoding. For instance, PyTorch’s chatbot tutorial demonstrates a multi-layer bidirectional GRU encoder. Its forward and reverse recurrent networks encode context from past and future directions, respectively. This is an instructional example, not evidence that a GRU is the best architecture for current chatbots.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11GRU vs. LSTM: what is the difference?
GRUs and long short-term memory (LSTM) units are both gated recurrent units: they regulate how information moves through a sequence. A GRU uses reset and update gates and, in the original paper’s characterization, is simpler to compute and implement than an LSTM unit.
Best Value
A 2014 evaluation compared GRUs, LSTMs, and traditional tanh recurrent units on polyphonic music and speech-signal sequence modeling. Its abstract reports that GRUs were comparable to LSTMs in those experiments, while the advanced gated units outperformed traditional tanh units. That is a historical result for particular tasks, not a general performance ranking. The cited sources do not establish a current, universal winner.
For a real application, compare the two models on the workload that matters. Useful criteria include:
- Validation performance on the target task and data.
- Parameter budget and memory use.
- Training and inference cost with the intended framework and hardware.
- Sequence length and the behavior that matters for the application.
- The exact framework implementation, including its gate equations.
The original GRU paper is available at arXiv:1406.1078, and the task-specific evaluation is described in arXiv:1412.3555.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
