Free tools Windows power users keep installed
One-click scans. No signup required.
In Keras’s supervised consistency-training example, a teacher first learns from clean, labeled images. A student then learns from augmented versions of those same images using both the ground-truth labels and the teacher’s predictions on the clean inputs. This teacher–student workflow aims to improve robustness to plausible image variations; it is not the same as unlabeled-data methods such as FixMatch.
How supervised consistency training works
The central idea is to encourage a classifier to make compatible predictions when an image is presented in its original form and in a transformed form. In the Keras example, the clean image has a known label, and the teacher’s prediction on it supplies an additional target for the student when it sees an augmented version.
As an Amazon Associate I earn from qualifying purchases.
- Train a teacher: fit an image classifier on clean, labeled training data with a supervised classification loss.
- Make clean-image targets: run the teacher on the clean images and retain the predictions, keeping each prediction paired with an augmented view of its original image.
- Create student inputs: apply a strong augmentation policy. The example uses RandAugment to produce noisy student inputs.
- Train the student: optimize it against both the dataset labels and the teacher targets for the corresponding clean images.
- Evaluate both settings: measure ordinary test-set performance and, separately, performance under corruptions or shifts relevant to the intended use.
The Keras walkthrough uses a custom loss that combines sparse categorical cross-entropy against labels with KL divergence between temperature-softened teacher and student logits. It averages the two loss terms. The temperature softens the distributions being compared; the loss therefore encourages the student’s output on the augmented image to align with the teacher’s output on its clean counterpart, while retaining direct supervision from the label.
What to adapt when implementing it
Keep each target aligned with its source image
The teacher target must belong to the same original image as the student’s augmented input. If batching, shuffling, caching or preprocessing causes images and targets to become misaligned, the consistency term trains the student toward unrelated predictions.
#1 Best Overall
Choose transformations that preserve the label
Consistency is useful only when the applied transformation produces a plausible variation that should retain the image’s class. An augmentation that changes the object’s identity, removes the evidence needed to classify it, or otherwise makes the label invalid can turn both the ground-truth and teacher targets into bad supervision.
Treat training settings as dataset-specific
The example includes learning-rate reduction and early-stopping callbacks in its teacher workflow, but its particular hyperparameters are not universal defaults. Validate augmentation strength, model capacity, target temperature and the balance implied by the loss on the dataset and deployment conditions that matter to you. The example starts from controlled initial weights and trains a student that is equal in size to or larger than the teacher.
Rank #2
- Use scikit-learn to track an example ML project end to end
- Explore several models, including support vector machines, decision trees, random forests, and ensemble methods
- Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection
- Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformers
- Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning
Use a custom loss rather than assuming a weight regularizer is the same thing
This method matches predictions across views of an image; it is not merely a penalty on model parameters. Keras’s Regularizer API is relevant when adding a parameter regularization penalty, but the teacher–student comparison in the example is implemented as a custom loss.
How to evaluate robustness without overstating results
Ordinary test accuracy and corruption robustness answer different questions. Report them separately if both matter: a strong result on the usual test distribution alone does not establish resilience to shifted inputs.
Rank #3
The Keras example discusses CIFAR-10-C, which it describes as covering 19 corruption types at five severity levels. Its short demonstration does not run the full benchmark assessment. The page notes a five-epoch demonstration, but it does not provide enough exposed experimental detail or exact results to support a general claim that the method improves accuracy by a particular amount. Treat it as an implementation walkthrough, not a quantified robustness result. For a defensible comparison, specify the dataset and splits, architecture, augmentations, training budget, baseline and evaluation protocol.
How it differs from FixMatch and AdaMatch
| Method | Unlabeled data required? | How targets are used | What distinguishes it |
|---|---|---|---|
| Supervised consistency example in Keras | No; it uses labeled images. | A teacher predicts clean images; the student matches those targets on augmented views and also learns from ground-truth labels. | Supervised teacher–student consistency with strong augmentation. Keras example |
| FixMatch | Yes; it is semi-supervised. | It forms pseudo-labels from weakly augmented unlabeled images and trains on strongly augmented versions when confidence exceeds a threshold. | Combines consistency regularization with confidence-filtered pseudo-labeling. Paper · Google Research summary |
| AdaMatch | It is a semi-supervision and domain-adaptation approach. | Its targets and training setup address labeled and unlabeled data across domains. | A related option when the problem includes unlabeled or shifted-domain data, rather than the algorithm implemented in the supervised Keras example. Keras AdaMatch example |
The supervised example is related to knowledge distillation and self-training: a teacher supplies targets that guide a student, with labels still contributing to the student’s objective. The Keras page traces the approach’s influences to FixMatch, Unsupervised Data Augmentation for Consistency Training and Noisy Student Training. The methods are related, but unlabeled data and confidence-thresholded pseudo-labels are central to FixMatch, not prerequisites for the supervised workflow described here. The Google Research FixMatch repository also states, “This is not an officially supported Google product.” Repository
Rank #4
Version and compatibility note
The Keras example page’s historical installation note says TensorFlow 2.4 or higher, while the page’s source has since been modified for newer Keras. Do not treat that old minimum as a current compatibility guarantee; check the example and your installed Keras, TensorFlow and backend versions before adapting code. Keras consistency-training example
Recommended Free Tools
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
