Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
This guide trains a task adapter for FacebookAI/roberta-base on labeled text-classification data using Hugging Face’s current adapters library. The standard workflow freezes RoBERTa’s pretrained weights and updates the adapter and classification head. The adapter is a small task-specific component, not a complete standalone model: keep track of the compatible base model and tokenizer as well.
What you are building
A bottleneck adapter is a small trainable module added to a pretrained transformer. RoBERTa supplies general language representations; the adapter learns task-specific changes, and a classification head turns the resulting representation into class logits.
In this tutorial, the intended trainable parts are the sentiment adapter and its prediction head. With the usual train_adapter() setup, the pretrained RoBERTa weights are frozen. You can keep several task adapters with a shared base model, but each adapter needs a compatible base checkpoint and, for classification, an appropriate head.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Adapters are a parameter-efficient option, not a guarantee of better results or faster wall-clock training. The original adapter paper reported GLUE results within 0.4 percentage points of full fine-tuning while adding 3.6% task-specific parameters per task in its experimental setup; that historical result does not predict performance on a different model, dataset, or configuration (original adapter research).
#1 Best Overall
- Materials and Design: The adapter is made with anti-interference zinc alloy metallic housing and minimalist design with anti-slippery embossments
- Connectors: Engineered for enhanced durability, the male USB C and female USB3 connectors are designed to be plugged and unplugged up to 10000 times
- Compatibility: This USB C to USB 3.0 adapter is compatible with iPhone 17/17e/17 Air/17 Pro/17 Pro Max and MacBook Pro after 2016 and MacBook Air after 2018 and most of the laptops, tablets and smartphones with a USB Type C port
- USB 3.0 Speed in Two: Came in two fast speed adapters in data transfer and charging with premium materials. A foam container is also included for storage and travel
- Compact and Easy to Use: Plug and play, no driver required; Simple structure, lightweight and portability; Also, you can sync or charge your phone with this USB C to USB adapter
Choose the right adaptation method
- Classic bottleneck adapter: Use the
adapterslibrary for modular task or language adapters, task heads, and adapter composition. This is the method used below. - LoRA and related methods: Use Hugging Face PEFT if you want LoRA, IA3, AdaLoRA, prefix tuning, or a broader PEFT workflow. It is a separate API and its checkpoints should not be treated as
adapterslibrary checkpoints. Transformers documents its PEFT integration and listspeft >= 0.19.1for the integration described there (Transformers PEFT integration). - Full fine-tuning: Choose this when updating all model weights is acceptable and modular task-specific artifacts are not important. It uses more trainable parameters and produces a full task-specific model checkpoint.
A task adapter is also different from a language or domain adapter: the latter is typically trained on language-modeling data to adapt representations, and is not by itself a drop-in classifier.
Prepare the environment
The AdapterHub project lists Python 3.9 or newer and PyTorch 2.0 or newer; check the project page for current requirements because they can change (Adapters project). A CPU can run a small demonstration, but practical training is generally more useful on a GPU. The frozen base model still occupies memory, and forward/backward activations also require memory.
python -m venv .venv
source .venv/bin/activate # macOS/Linux
# .venvScriptsactivate # Windows
python -m pip install -U pip
pip install -U adapters datasets evaluate accelerate scikit-learnFor a reproducible project, record the installed package versions and use a tested, pinned environment. Current code examples and older online tutorials can differ in method and argument names. The adapters package is the current library; the older adapter-transformers ecosystem used imports such as AutoModelWithHeads. AdapterHub documents the newer package and its compatibility and migration context (Hugging Face Hub: Adapters; AdapterHub documentation).
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchLoad and inspect a labeled dataset
This example uses IMDb reviews, whose dataset fields include review text and a binary label. Its test split is used below as the evaluation split for brevity. When selecting hyperparameters, use a validation split; reserve a held-out test set for the final assessment rather than tuning against it.
Rank #2
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
from datasets import load_dataset
dataset = load_dataset("imdb")
print(dataset)For your own CSV files, use separate training and validation files (and a separate test file if available):
dataset = load_dataset(
"csv",
data_files={
"train": "train.csv",
"validation": "validation.csv",
"test": "test.csv",
},
)The preprocessing below expects a text column named text and an integer label column. Change examples["text"] if your column is named, for example, review or sentence. Labels should consistently map to class IDs beginning at zero for this example. Check for missing labels and confirm the mapping before training; for binary sentiment, the code assumes 0 means negative and 1 means positive.
Tokenize the examples
Use the same RoBERTa identifier for the tokenizer and model. Truncation prevents overlong inputs from exceeding the selected limit. The value 256 below is only a starting point: a longer limit can preserve more text while increasing memory use and time, and a shorter one may discard useful context. Dynamic batch padding avoids padding every example to the same global length.
Recommended Free Tools
from transformers import AutoTokenizer
model_name = "FacebookAI/roberta-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
def preprocess_function(examples):
return tokenizer(
examples["text"],
truncation=True,
max_length=256,
)
tokenized_dataset = dataset.map(
preprocess_function,
batched=True,
remove_columns=["text"],
)For sentence-pair classification, pass both text fields to the tokenizer:
Rank #3
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
def preprocess_function(examples):
return tokenizer(
examples["sentence1"],
examples["sentence2"],
truncation=True,
max_length=256,
)Transformers’ sequence-classification workflow likewise maps tokenization over the dataset and uses a data collator to pad batches (sequence classification guide).
Add a task adapter and classification head
Load RoBERTa as an adapter-aware model, add a Pfeiffer bottleneck adapter, and create a two-class head. The adapter and head are both required for the intended classifier; set the head active so the forward pass uses it. Adapter APIs can change across releases, so use the matching current documentation if your installed version reports a different head signature.
from adapters import AutoAdapterModel
adapter_name = "sentiment"
head_name = "sentiment_head"
model = AutoAdapterModel.from_pretrained(model_name)
model.add_adapter(adapter_name, config="pfeiffer")
model.add_classification_head(
head_name,
num_labels=2,
id2label={0: "NEGATIVE", 1: "POSITIVE"},
)
model.active_head = head_nameThe RoBERTa sequence-classification setup pairs the encoder with a task-specific head that produces logits (RoBERTa model documentation).
Freeze the base model and audit trainable parameters
Call train_adapter() before training. It freezes the model’s other parameters and enables training for the selected adapter; the head must remain trainable too. Activate the adapter for the forward pass.
Rank #4
- PACK OF 2 & GREAT VALUE：Package includes 2pcs dual port wall charger enabling you keep one at home, one at work and one for traveling. Great valued alternatives to the brand. Various vibrant colors available to easier to identify which one is for your gadgets
- WIDE COMPATIBILITY：Usb c charging block is widely compatible with iPhone 14/14 Plus/14 Pro/14 Pro Max/iPhone 13/13 Pro Max/iPhone 12/12 Mini/12 Pro/12 Pro Max/iPhone11/11 pro/11pro max /XS/XS Max/XR/X/8/7/6, iPad Pro 11"2020/iPad Air 3 10.5" and more latest smartphones and tablets
- EFFICIENT CHARGING：Charging wall adapter that delivers a sturdy full power for efficient charging, Allowing you to quickly charge your devices especially when people in a hurry
- SMART SAFE GURAD IN CHARGING：Usb-c wall charger also includes an intelligent chip that safeguards your phone against overheating, overvoltage, and general electrical surges. You will not regret getting this charging block for the best charging performance
- DUAL PORT YET COMPACT：Type c charging block with dual port in a single plug gives you the flexibility to use an older USB-A cable as well as the USB-C cable. It is also made into a compact cube that doesn’t take much spaces. Perfect for tight places or carry on the go
model.train_adapter(adapter_name)
model.set_active_adapters(adapter_name)
model.active_head = head_name
def trainable_parameters(model):
total = 0
trainable = 0
for parameter in model.parameters():
count = parameter.numel()
total += count
if parameter.requires_grad:
trainable += count
return trainable, total
trainable, total = trainable_parameters(model)
print(f"Trainable: {trainable:,}")
print(f"Total: {total:,}")
print(f"Percent: {100 * trainable / total:.2f}%")The reported percentage varies with the adapter configuration and bottleneck size, model size, head, and whether other modules such as embeddings are enabled for training. If nearly all parameters are trainable, check that the model came from AutoAdapterModel and that train_adapter() ran.
Train and evaluate
The following uses accuracy and binary F1. Accuracy alone can be misleading for imbalanced labels. For multiclass tasks, choose macro F1 when each class should count equally, or weighted F1 when class prevalence should influence the aggregate.
import numpy as np
import evaluate
from adapters import AdapterTrainer
from transformers import TrainingArguments, DataCollatorWithPadding
accuracy = evaluate.load("accuracy")
f1 = evaluate.load("f1")
def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = np.argmax(logits, axis=-1)
return {
"accuracy": accuracy.compute(
predictions=predictions,
references=labels,
)["accuracy"],
"f1": f1.compute(
predictions=predictions,
references=labels,
average="binary",
)["f1"],
}
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
training_args = TrainingArguments(
output_dir="roberta-sentiment-adapter",
learning_rate=1e-4,
per_device_train_batch_size=16,
per_device_eval_batch_size=16,
num_train_epochs=3,
weight_decay=0.01,
eval_strategy="epoch",
save_strategy="epoch",
load_best_model_at_end=True,
report_to="none",
)
trainer = AdapterTrainer(
model=model,
args=training_args,
train_dataset=tokenized_dataset["train"],
eval_dataset=tokenized_dataset["test"],
processing_class=tokenizer,
data_collator=data_collator,
compute_metrics=compute_metrics,
)
trainer.train()
print(trainer.evaluate())The learning rate, batch sizes, and three epochs are starting values, not universal settings. Batch size depends on available memory and sequence length; learning rate and epoch count depend on the dataset and can lead to underfitting or overfitting. The current AdapterHub training guide describes AdapterTrainer and the adapter-training workflow (AdapterHub training documentation). Recent Transformers examples use names such as eval_strategy and processing_class; older versions used alternatives such as evaluation_strategy and tokenizer. Consult documentation for the installed version if an argument is rejected.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Save the adapter and reload it
Export the task adapter with its head, and save the tokenizer alongside it. This adapter export is distinct from a trainer checkpoint, which may also contain optimizer, scheduler, and trainer state for resuming training.
Best Value
- Anker Advantage: Join the 55 million+ powered by our leading technology.
- Widely Compatible: Transform any USB-C port into a USB-A port and connect up a wide range of USB-A devices including external hard drives, phones, mice, printers, and more.
- Strong and Stylish: Finished in Space Gray and constructed from premium scratch-resistant aluminum, the adaptor not only blends seamlessly with your MacBook Pro but also withstands the wear and tear of day-to-day use.
- Superior Connectors: Engineered for enhanced durability, the male USB-C and female USB-A 3.0 connectors are designed to be plugged and unplugged up to 10,000 times—basically for life.
- Space for Two: The ultra-slim form factor ensures there’s space to plug two adaptors side by side into your MacBook Pro’s USB-C ports.
model.save_adapter(
"sentiment_adapter",
adapter_name,
with_head=True,
)
tokenizer.save_pretrained("sentiment_adapter")with_head=True includes the classification head needed for this task-specific inference. If you deliberately manage a shared head separately, save and restore that head through the API supported by your installed release.
To reload a locally saved adapter in a clean process:
import torch
from adapters import AutoAdapterModel
from transformers import AutoTokenizer
model_name = "FacebookAI/roberta-base"
tokenizer = AutoTokenizer.from_pretrained("sentiment_adapter")
inference_model = AutoAdapterModel.from_pretrained(model_name)
inference_model.load_adapter("sentiment_adapter", set_active=True)
inference_model.eval()
text = "The product was easy to use and worked well."
inputs = tokenizer(text, return_tensors="pt", truncation=True)
with torch.no_grad():
outputs = inference_model(**inputs)
prediction = outputs.logits.argmax(dim=-1).item()
print(inference_model.config.id2label[prediction])For CPU/GPU consistency, load the model and place both model and inputs on the same device. If the adapter directory or head does not restore as expected, consult the documented local and Hub loading forms for the package version in use (adapter loading and Hub documentation).
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsWhen publishing an adapter, include its base model identifier, RoBERTa variant, adapter configuration, label mapping, tokenizer settings, maximum sequence length, training details, data provenance, license, and intended-use limitations. The Hub documentation describes push_adapter_to_hub() for publishing adapter artifacts and metadata.
Quick Recap
Troubleshoot common problems
- Old import such as
AutoModelWithHeadsoradapter-transformers: The example targets the legacy ecosystem. Use the currentadapterspackage and adapter-aware model rather than mixing old fork imports with current Transformers code. The project documentation covers migration context (AdapterHub documentation). train_adapter()is missing: Check that you installedadapters, loaded the model withAutoAdapterModel, and are not using a PEFT model with a different API.- No active head or unexpected logits shape: Verify the active head,
num_labels, label column, and integer label mapping. Multilabel classification requires a different loss and thresholding approach from the single-label example. - Training runs but quality is poor: Check label mapping, duplicates or leakage, class balance, sequence truncation, adapter activation, head trainability, and whether validation data resembles production inputs. A small subset that can be deliberately overfit is useful for checking the training path.
- CUDA out of memory: Reduce per-device batch size or maximum length, use gradient accumulation, consider supported mixed precision or gradient checkpointing, choose a smaller checkpoint, and retain dynamic padding. Adapters reduce trainable parameters and optimizer state; they do not eliminate base-model or activation memory.
- Adapter will not load later: Verify the compatible base checkpoint, adapter weights and configuration, tokenizer, and classification head are available. A RoBERTa-base adapter is not automatically compatible with RoBERTa-large, BERT, DeBERTa, or XLM-RoBERTa.
- Runs differ: Record seeds, dataset splits and preprocessing, package versions, hardware/CUDA details, and precision settings. Use multiple seeds or uncertainty estimates for serious comparisons.
Checklist before using the classifier
- Record the base model identifier and exact adapter configuration.
- Keep training, validation, and final test data roles separate.
- Confirm class IDs and human-readable label names match at training and inference.
- Test loading the exported adapter and tokenizer in a fresh process.
- Review the base model and dataset licenses, privacy constraints, and intended-use limits.
- Monitor performance on representative production data; an adapter does not remove domain-shift or bias risks.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
