The Neural Network Zoo is a visual guide to neural-network architectures: it maps influential designs and shows how they relate, from feed-forward networks and CNNs to recurrent models, autoencoders, GANs and attention-based systems. Created by Fjodor van Veen for the Asimov Institute in 2016 and later developed into an academic overview with Stefan Leijnen, it is useful for learning the field’s vocabulary and history—not as a complete or up-to-date catalog of every model.
What is The Neural Network Zoo?
The Neural Network Zoo is both an educational web article and a diagram of neural-network architectures. Its purpose is to make a crowded field of names and acronyms easier to navigate by grouping designs according to structure, connections, historical relationships and, in some cases, intended tasks. The original Asimov Institute page was posted on September 14, 2016. A later proceedings paper by Stefan Leijnen and Fjodor van Veen, published May 12, 2020, formalized the project as a taxonomy for comparing architectures, chronology and lines of influence.
As an Amazon Associate I earn from qualifying purchases.
The web page records a notable update on April 22, 2019, which added Capsule Networks, Differentiable Neural Computers and Attention Networks and removed Support Vector Machines. Although the page currently displays a January 3, 2025 modification date, that metadata alone does not establish that the taxonomy was comprehensively refreshed for 2025 or 2026. Its creators say a complete list is practically impossible because new designs keep appearing.
Use the Zoo to recognize architectural ideas, understand broad historical connections and find links to original papers. It is not a benchmark, implementation tutorial, model-selection guide or comprehensive survey of modern foundation models. Its categories are a helpful map, not an official standard: labels placed side by side may describe a whole family, a component, a connection pattern or a training approach.
#1 Best Overall
How to read the diagram
Start with the connections, then ask what the picture leaves out. A diagram can show how information might move through a network, but it cannot by itself tell you the full training objective, data assumptions, optimization behavior or deployment characteristics.
- Direction: One-way paths generally indicate feed-forward computation; loops suggest recurrence or feedback.
- Connection pattern: Dense connections link many units, local connections suggest convolution, and skip paths let information bypass layers.
- Memory: A model may rely on activations, recurrent hidden state, gated cell state, a latent variable, explicit external memory or associative dynamics.
- Multiple components: A GAN diagram, for example, represents a generator and discriminator trained together, not just one network.
- Training objective: Classification, reconstruction, generation, prediction and self-organization can produce different behavior even when diagrams look similar.
For a more detailed explanation of the poster’s visual conventions and limitations, see the companion overview.
Major architecture families in the Zoo
| Family | Defining idea | Typical fit | Important limitation |
|---|---|---|---|
| Feed-forward | Directed, acyclic computation through layers | General vector inputs; classification or regression | No built-in mechanism for sequence memory or spatial locality |
| Convolutional neural network (CNN) | Local filters with shared weights | Images, grids and other structured signals | Its spatial inductive bias may not suit every input |
| Recurrent neural network (RNN) | Earlier steps influence later computation through hidden state | Ordered sequences | Sequential computation and long-range training difficulties |
| LSTM or GRU | Gated recurrent cells regulate information flow | Sequence tasks needing controlled state | More complexity and computation than a basic recurrent cell |
| Autoencoder | Encode an input and reconstruct it | Representation learning, compression or anomaly detection | Reconstruction quality does not guarantee a useful representation |
| Variational autoencoder (VAE) | Learn a probabilistic latent representation that can be sampled | Generative modeling and latent-variable learning | Its objective and output behavior differ from a plain autoencoder |
| Generative adversarial network (GAN) | Generator and discriminator train in opposition | Generating synthetic data | Training can be unstable or suffer mode collapse |
| Residual network | Shortcut paths bypass one or more layers | Training deeper networks | It is a connectivity strategy that can be combined with other families |
| Attention and Transformers | Content-dependent weighting of information; Transformers center attention rather than recurrence | Contextual modeling of sequences and other inputs | Compute and memory costs can grow with model and input size |
| Neural Turing Machine / Differentiable Neural Computer | Neural controller interacts with explicit external memory | Research on memory-intensive or algorithmic tasks | Specialized and operationally complex; inclusion does not imply widespread deployment |
| Capsule network | Vector-valued capsules and routing aim to preserve richer feature properties | Research into perceptual representations | Not a settled replacement for CNNs |
| Self-organizing map (SOM) | Competitive learning organizes nearby units around a best-matching unit | Exploration and visualization of unlabeled data | Not interchangeable with supervised deep-learning models |
| Hopfield network | Recurrent associative memory represented through network dynamics | Historical and conceptual study of memory models | The name covers distinct formulations, so the specific version matters |
Feed-forward networks: the baseline
Perceptrons, multilayer perceptrons and radial-basis-function networks belong to the broad feed-forward family. Computation travels from input toward output without cycling back. Multilayer networks can approximate complex functions, but the architecture itself does not encode a sequence’s order or an image’s spatial relationships. Backpropagation is a common way to adjust weights using prediction error; it is a training method, not an architecture.
Recommended Free Tools
Convolutional networks: local patterns and shared filters
A convolutional layer applies learned filters across local regions, reusing the same weights at different positions. This can capture recurring patterns without learning a separate connection for every input location. Pooling and striding reduce spatial resolution. CNNs are associated with images, but convolution also applies to audio, video, time series and scientific grids when local structure matters.
Rank #2
Recurrent networks, LSTMs and GRUs: state across steps
An RNN carries information from earlier steps into later computation through a hidden state. Bidirectional variants process a sequence in both directions; stacked variants place recurrent layers on top of one another. LSTMs and GRUs are gated recurrent cell designs that regulate what information is retained or updated. They were developed to improve learning over time, not to eliminate every long-range dependency problem; recurrent computation can still be difficult to train over long sequences and is inherently stepwise.
Autoencoders and VAEs: reconstruction versus sampling
An autoencoder maps an input into a representation and learns to reconstruct it. A variational autoencoder instead learns a probabilistic distribution over latent representations, making it possible to sample latent values for generation. The two may have similar encoder-decoder diagrams, but a VAE’s probabilistic formulation and training objective change what it learns and how it is used. An encoder-decoder is not automatically a VAE.
GANs: two networks, one adversarial training system
A GAN pairs a generator, which produces samples, with a discriminator, which tries to distinguish generated samples from real ones. Their objectives compete during training, so “GAN” refers to a training framework involving two models as well as their architecture choices. Instability, sensitivity to the balance between the two networks and mode collapse are familiar difficulties. DCGAN is a convolution-focused GAN variant; GANs are not limited to image generation.
Residual connections: a pattern that crosses families
A residual or skip connection lets information and gradients bypass one or more layers. This helps make deeper stacks easier to optimize. A residual network is not necessarily separate from a CNN or feed-forward network: residual connections can modify those architectures. The 2020 paper discusses residual networks as feed-forward designs whose connections can cross multiple hidden layers.
Rank #3
Attention and Transformers: selecting context
Attention lets a model weight information from other positions or states. It may be added to recurrent encoder-decoder systems, or used as the central computational mechanism in a Transformer, where recurrence is not the primary sequence-processing operation. Self-attention, cross-attention and spatial or temporal attention are related ideas, not a single interchangeable mechanism. The Zoo’s attention category is a useful starting point, not a complete taxonomy of today’s Transformer, multimodal or foundation-model designs.
External memory: Neural Turing Machines and DNCs
Ordinary recurrent models store context in hidden state. Neural Turing Machines and Differentiable Neural Computers add an explicit memory structure that a controller can read and write through differentiable operations. The Zoo describes DNCs as using a recurrent controller, scalable external memory and multiple attention mechanisms for memory access. These models are conceptually important for connecting neural computation and explicit storage, but their presence on the poster does not establish them as default production architectures.
Capsules: preserving richer feature properties
Capsule networks group features into vectors rather than representing each detected feature only as a scalar activation. The proposal aims to preserve properties such as pose or orientation, with dynamic routing used to organize relationships among capsules. This is a research direction motivated by limitations attributed to pooling, not an established universal replacement for convolutional networks.
Self-organizing maps: competitive learning
A Kohonen network, or self-organizing map, uses competitive learning: units compete to match an input, and the best-matching unit and its neighbors are adjusted. The result is a topologically organized representation that can support exploration and visualization of unlabeled data. Its learning setup differs from conventional supervised training, so it should not be treated as another interchangeable deep classifier.
Hopfield networks: associative memory
Hopfield networks are recurrent or energy-based associative-memory models in which network dynamics can retrieve stored patterns. The name spans different generations of models, including classical discrete or continuous formulations and later variants. When using the Zoo as a historical map, identify which formulation a paper or implementation means rather than assuming one fixed design.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Why similar diagrams can mean different things
Architecture names operate at different levels. Confusing these levels is one of the easiest ways to misread the poster:
- Backpropagation is an optimization procedure, not a network family.
- Attention is a mechanism that can be placed inside different architectures; a Transformer makes it central.
- Residual describes a connection pattern that may be layered onto another family.
- VAE refers to a probabilistic latent-variable approach and its associated model pattern, not any encoder-decoder.
- GAN describes adversarial training involving a generator and discriminator, not merely a single image-making network.
Likewise, a simple-looking diagram does not make a model interpretable. A visualization of attention weights alone does not establish that those weights are a causal explanation of a prediction.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHow to use the Zoo when choosing an architecture
The diagram can help generate candidates, but a sensible choice starts with the task and constraints rather than with an acronym.
Best Value
- Identify the data structure. Images and grids may benefit from local convolution; ordered sequences may suit recurrence, attention or both; relational data may need graph-specific message passing.
- Define the objective. Decide whether the system predicts labels or values, reconstructs inputs, generates samples, learns representations, retrieves information or controls actions.
- Ask how far context must travel. Local filters emphasize neighborhoods, recurrent models pass state step by step, attention links positions selectively, and external memory separates storage from the controller.
- Set compute and latency constraints. Consider training parallelism, sequential inference dependencies, memory use, hardware support and response-time requirements.
- Check maturity and tooling. Available pretrained models, implementation quality, monitoring needs and team experience can matter more in production than historical novelty.
- Evaluate empirically. Architecture alone does not determine performance; data, objective, optimization, scale, regularization, implementation and evaluation design all matter.
The right answer may combine categories: a model can be convolutional, residual, attention-augmented and generative at once. Treat the Zoo’s lines as conceptual relationships, not a definitive genealogy accepted by every researcher.
What the Zoo leaves out—and why it remains useful
The creators explicitly say the list is not comprehensive. The project is strongest as a historical and conceptual map of influential designs, especially architectures developed or popularized in the period it covers. It should not be mistaken for a current catalog of every graph, state-space, mixture-of-experts, retrieval-augmented, diffusion or foundation-model system. Those areas and their combinations require more specialized, current references.
That limitation does not make the Zoo obsolete. It remains useful for learning recurring ideas—locality, recurrence, memory, latent variables, adversarial objectives and attention—and for tracing how later designs build on or recombine them. For the original visual and linked papers, visit the Asimov Institute resource; for its academic framing, read Leijnen and van Veen’s 2020 paper.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
