Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsEANet, in the Keras context, is the External Attention Transformer: an image classifier that replaces the usual self-attention step with two small, shared learnable memories. The official Keras example trains it on CIFAR-100, a 100-class set of 32×32 colour images, and the example shows how the whole pipeline fits together in Python. This guide walks through that example, explains what external attention changes and why it matters, and flags what the example does and does not prove.
What EANet refers to in this Keras example
The acronym has been used for more than one architecture, so it helps to anchor the term. Here, EANet means the External Attention Transformer described in the Keras code example “Image classification with EANet (External Attention Transformer),” which is listed at keras.io/examples/vision/eanet/. The page’s introduction describes the core idea in its own words:
As an Amazon Associate I earn from qualifying purchases.
“EANet introduces a novel attention mechanism named external attention, based on two external, small, learnable, and shared memories, which can be implemented easily by simply using two cascaded linear layers and two normalization layers.”
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →The example is authored by ZhiYong Chang. The page lists its creation date as 2021-10-19 and its last modification as 2023-07-18. Because the page does not pin a Keras release, treat the code as a reference implementation to check against your installed version rather than as a guarantee that it runs unchanged on every current release.
#1 Best Overall
The task: CIFAR-100 image classification
The example uses CIFAR-100 with the standard split of 50,000 training images and 10,000 test images. Each image is 32×32 pixels with three colour channels (RGB), and labels span 100 classes. That makes the input shape (32, 32, 3) and the output a 100-way softmax.
Small images keep the experiment cheap to run, but they also make the patch arithmetic easy to follow. Splitting a 32×32 image into 2×2 patches produces 16 × 16 = 256 patches per image, which is the sequence length the transformer blocks process.
How the model is assembled
The example follows a conventional vision-transformer layout, with the attention step as the one part that changes. The order of operations is:
- Load and prepare the data. Import
keras,layers, andops, load CIFAR-100, and one-hot encode the labels for 100 classes. - Augment the images. The example defines a data augmentation stage that runs on training images before they reach the network.
- Extract patches. Each augmented image is cut into 2×2 patches, giving 256 patch vectors per image.
- Embed the patches. Each patch vector is projected to an embedding dimension of 64, and the sequence is passed forward.
- Apply transformer encoder blocks. The example stacks eight encoder blocks. Each block uses the selected attention type, which for this example is external attention, followed by the projection and feed-forward layers of a standard transformer block.
- Pool and classify. Global average pooling collapses the sequence into one vector, and a dense softmax layer with 100 units produces class probabilities.
Self-attention versus external attention
Standard self-attention lets every patch attend to every other patch, so the cost grows with the square of the sequence length. The example’s own complexity account puts self-attention at O(d·N²) and external attention at O(d·S·N), where N is the number of positions being attended over (here, the patch sequence), and d and S are hyperparameters.
| Attention type | Complexity stated in the example | Main parameters | Where the cost scales with N |
|---|---|---|---|
| Self-attention | O(d·N²) | d (hyperparameter), N (sequence length) | Quadratically |
| External attention | O(d·S·N) | d and S (hyperparameters), N (sequence length) | Linearly, for fixed S |
The “two external, small, learnable, and shared memories” in the quoted introduction are the S-dimensional memory structures that replace direct patch-to-patch comparison. Because the memories are learned and shared across the dataset rather than computed from each image, the attention step depends on the sequence length only linearly in this formulation.
These are asymptotic statements from the example’s explanation. They describe how cost grows, not how long a training run takes on your hardware, and the example does not report a measured runtime or accuracy comparison against self-attention.
Configuration values used in the example
The example sets the following values. They reproduce the example’s configuration; they are not general recommendations, and you will likely change some of them for your own dataset or compute budget.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware match| Setting | Value in the example |
|---|---|
| Patch size | 2×2 |
| Patches per image | 256 |
| Embedding dimension | 64 |
| Attention heads | 4 |
| Transformer blocks | 8 |
| Batch size | 128 |
| Epochs | 50 |
| Learning rate | 0.001 |
| Weight decay | 0.0001 |
| Label smoothing | 0.1 |
| Attention dropout | 0.2 |
| Projection dropout | 0.2 |
The training setup uses categorical cross-entropy with label smoothing, weight decay, and a validation split, together with the batch size and epoch count above.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Running the example and adapting it
To work through the example yourself, start from the page at keras.io/examples/vision/eanet/, which contains the full code. Then check the following before you train:
- Confirm your Keras version. Run
python -c "import keras; print(keras.__version__)"and compare it with the Keras release you are targeting. The example does not pin a version, so an API change in a newer release can break a line that worked when the page was written. - Keep the patch arithmetic consistent. If you change the input resolution, recompute the patch count. A 64×64 input with 2×2 patches gives 32 × 32 = 1,024 patches, which changes the sequence length and the cost.
- Expect longer runs on larger inputs. Because the example’s complexity grows with the number of patches, scaling up the resolution affects training time even though external attention scales linearly rather than quadratically.
- Reconsider the schedule for your dataset. The 50-epoch, batch-size-128 setting suits the example’s CIFAR-100 run. Adjust epochs, batch size, and learning rate against your own validation curves rather than copying them.
What the example does not establish
The example demonstrates an implementation pattern and a set of configuration values. It does not publish a final test accuracy or a head-to-head benchmark against other models in the reviewed material, so it does not tell you how EANet ranks against a self-attention transformer or a convolutional network on CIFAR-100. If you need those numbers, train each model under the same split, input size, hardware, schedule, and parameter budget, and measure accuracy and inference latency yourself.
The page’s creation and modification dates (2021-10-19 and 2023-07-18) show when the example was written and last changed. Anything that depends on a particular Keras or TensorFlow release should be checked against that release’s documentation.
Quick Recap
“
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
