Active learning for text classification is a human-in-the-loop cycle: train a model on a small labeled set, ask for labels on selected examples from a larger unlabeled pool, add those labels, and retrain. Keras’s review-classification tutorial demonstrates that process on IMDB sentiment data; it is an example of one sampling approach, not proof that active learning always beats random selection or lowers labeling costs.
How pool-based active learning works
In pool-based active learning, you begin with a small set of labeled examples and a larger collection of unlabeled text. A classifier learns from the labeled examples, then a query strategy chooses which pool items would be useful to label next. A human annotator supplies those labels, the newly labeled items join the training set, and the model is retrained.
The Keras tutorial calls the annotator an “oracle,” defining it as “an annotator that cleans, selects, labels the data, and feeds it to the model when required.” In practice, that role may be performed by one reviewer or by a labeling team. The process repeats until a chosen performance or business target is met, or the available data or labeling budget runs out.
What the Keras review-classification example does
Keras’s Review Classification using Active Learning, by Darshan Deshpande, was created in 2021 and last modified on 2024-05-08. It uses IMDB review sentiment data and combines the TensorFlow Datasets training and test splits for its tutorial experiment.
#1 Best Overall
- Use scikit-learn to track an example ML project end to end
- Explore several models, including support vector machines, decision trees, random forests, and ensemble methods
- Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection
- Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformers
- Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning
50,000 reviews — the combined IMDB setup used in the Keras tutorial; this is dataset context, not evidence of a performance gain.
The example turns review text into integer sequences with Keras TextVectorization, then passes those sequences to an embedding-based neural classifier. It separates seed training data, validation data, test data, and an unlabeled pool. The binary classifier uses binary cross-entropy and tracks binary accuracy, false negatives, and false positives.
Its sampling rule
The tutorial derives a positive-versus-negative sampling ratio from the false-negative and false-positive counts measured on its test set. It then samples from class-separated pools, adds selected examples to the training data, and repeats training. This is the tutorial’s particular design, not a general default for active learning.
The example also discusses uncertainty sampling and mentions committee sampling, entropy-based sampling, and minimum-margin sampling. Its split sizes, vocabulary settings, sequence length, batch size, and iteration settings are choices made for the demonstration; they should not be treated as universal Keras settings.
How to choose a query strategy
No single query strategy is best for every text-classification task. Compare options against the labeling workflow, available model outputs, and the kind of examples you need:
| Decision axis | What to consider |
|---|---|
| Uncertainty or informativeness | Does the method prioritize examples the model finds hard to classify? Least-confidence, entropy, and margin-based methods are examples of this family. The Keras tutorial and the Google Research active-learning repository describe examples of uncertainty-based approaches. |
| Diversity and redundancy | Will a batch contain different kinds of reviews, or many near-duplicates? The Google Research repository describes k-center-greedy as selecting representative points to reduce the maximum distance to a labeled point. This can complement uncertainty-based selection. |
| Batch or sequential selection | Does the strategy choose several examples before receiving any new labels, or update its choices after each label? The Keras tutorial samples batches. The modAL documentation discusses configurable query strategies and batch construction. |
| Model and data compatibility | Some strategies require class probabilities, uncertainty estimates, or gradients. Confirm that your classifier can provide the information a chosen method needs; the available documentation does not establish a complete compatibility matrix for every model and strategy. The modAL project documents combining Keras models with custom query strategies and uncertainty measures. |
| Labeling and compute budget | Weigh the likely value of each queried label against human review time, retraining cost, and the need to keep evaluation data representative. The cited examples do not establish a general price or annotation-savings figure. |
Evaluate the process without contaminating the test set
Keep a representative, held-out evaluation set separate from the unlabeled query pool. The Keras tutorial emphasizes careful test sampling and tracks false positives and false negatives, but its example is not a controlled, general demonstration that active learning improves results.
Because the tutorial’s sampling ratio uses false-negative and false-positive counts from its test set, do not copy that choice into a production evaluation workflow. Repeatedly steering training or query decisions with the final test set makes it part of model development. Instead, use a validation or query-selection signal during iteration and reserve a final untouched test set for the end.
Measure outcomes on your own data and against your own baseline—often random sampling—using the metric and labeling budget that matter to the task. The tutorial does not establish a general accuracy gain, quantified reduction in annotation, or universal advantage over random selection.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsRunning the example in a current Keras environment
The example sets the Keras backend to TensorFlow. The tutorial page and the Keras 3 API documentation do not establish a tested compatibility matrix for the tutorial’s Python, Keras, TensorFlow, and dependency versions. Check and record the versions in your environment before relying on a copied notebook; the current API overview is not a compatibility test for this particular example.
When active learning is worth trying
Active learning is worth evaluating when you have a sizeable unlabeled text pool, a reliable way to label examples, and a measurable reason to prioritize some labels over others. It does not eliminate annotation: it changes which examples are sent to annotators. Compare its results with a simple baseline, keep evaluation data independent, and account for both human and compute costs before adopting it.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
