Perturbation probing is a proposed way to investigate whether specific behaviors in an aligned language model depend on identifiable feed-forward network (FFN) neurons. In experiments reported by Liu, Li and Wu, some behaviors were concentrated in small sets of neurons, but other circuits varied across architectures or resisted the same intervention. The results are evidence about particular models, behaviors and benchmarks—not a universal measure of safety.
What perturbation probing does
The method is designed to form and test causal hypotheses about a targeted behavior inside a model. Rather than judging only the model’s visible responses, it looks for FFN neurons that may contribute to that behavior and tests candidates by intervening on them. The paper describes two forward passes per prompt to generate task-specific hypotheses, without backpropagation, followed by an intervention sweep that the abstract describes as about 150 intervention passes amortized across identified neurons. The two-pass figure is per prompt for the hypothesis-generation step; it is not the total number of passes for an entire study.
As an Amazon Associate I earn from qualifying purchases.
In this context, an intervention can test whether changing or ablating a candidate neuron changes the measured behavior. That makes perturbation probing an internal, model-specific diagnostic—not simply another prompt-based test. The authors report studying eight behavioral circuits across 13 models and four architecture families. Their paper, “Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs”, was submitted to arXiv on April 30, 2026.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsWhat the reported experiments found
The figures below describe the experiments reported by Liu, Li and Wu. Their denominators and endpoints differ, so they should not be combined into a single safety score.
| Behavior and model | Reported intervention result | What the result measures |
|---|---|---|
| Refusal template, Qwen3-4B | About 50 neurons—0.014% of all neurons in the model—were implicated in controlling the safety refusal template. Ablating them changed the response format on 80% of 520 AdvBench prompts. The authors report three harmful-compliance cases, all with disclaimers. | The 80% figure is about response format, not the percentage of safety removed. The three compliance cases are a separate outcome and do not show that changed wording caused harmful compliance. |
| Multi-turn sycophantic capitulation, Qwen3.5-2B | The initial behavior occurred on 36.7% of 30 questions. Ablating 20 neurons reduced the reported capitulation to zero in that experiment. | A result for the tested multi-turn questions and model, not a claim that sycophancy was eliminated generally. |
| Factual correction | Amplifying 10 related neurons raised factual correction from 52% to 88% across 200 TruthfulQA prompts. | A change in the measured correction outcome on that prompt set; it does not establish a general improvement in truthfulness. |
| Language selection | The paper reports 99.1% on 580 language-selection prompts. Direction injection switched English output to Chinese in three of 19 tested models for this circuit. | The intervention’s success was architecture- and model-dependent. The authors report failure in the other 16 models and on math, code and factual circuits. |
The language-selection result came with observed conditions: bilingual training, an FFN-to-skip signal ratio between 0.3 and 1.1, and linear representability of the circuit. Those conditions help explain why a strong result on one prompt set does not imply that the same steering method will work across models or behaviors.
Why circuit structure matters
The paper distinguishes “opposition circuits,” which appear when reinforcement learning from human feedback (RLHF) suppresses a tendency from pre-training, from routing circuits associated with pre-training behaviors distributed through attention. The authors also report architectural variation: a concentrated FFN bottleneck in Qwen models and a normalization-shielded circuit in Gemma.
Rank #2
Across the 13 models, the authors say the FFN-to-skip signal ratio helped distinguish circuit structures and predict an appropriate intervention. That is a proposed circuit-structure and intervention diagnostic. The reported work does not validate the ratio as a comprehensive measure of real-world safety or as a predictor of overall safety in production.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Does this show that LLM safety is fragile?
It shows that particular behaviors in particular tested models can be sensitive to interventions on particular internal components. The Qwen3-4B refusal experiment is a useful example of why the endpoint matters: ablating the implicated neurons changed response formatting on many prompts, but the authors separately report only three harmful-compliance cases, each with a disclaimer. A refusal’s wording and a model’s willingness to provide harmful assistance are related evaluation concerns, but they are not interchangeable measurements.
Likewise, reducing sycophantic capitulation in one Qwen experiment and increasing factual correction on a TruthfulQA set show that internal interventions can alter targeted behaviors in those conditions. They do not establish that the same neurons, effects or weaknesses occur across models, versions, prompts, deployments or adversaries. The study’s own cross-model differences—especially the language-steering failures—argue against treating a successful intervention as a universal property.
How to interpret it alongside other evaluations
Perturbation probing asks an internal-circuit question; external evaluations ask how a model responds under specified prompts or test conditions. Neither perspective alone answers every safety question. The paper does not establish that its method outperforms a comprehensive set of red-team or benchmark approaches. When evaluating what it adds, check:
Rank #4
- Access and compute: Can the evaluator inspect model internals and intervene on weights or activations? A prompt-only evaluator cannot directly reproduce an internal intervention study.
- Behavior and architecture coverage: Which behaviors, models and architecture families were actually tested? A finding for a refusal template or language-selection circuit should not be generalized to untested behaviors.
- Benchmark and sample scope: What prompts and sample sizes underpin the result? The 30-question sycophancy experiment and the 520-prompt AdvBench experiment answer different, bounded questions.
- Measured endpoint: Is the reported change in refusal wording, harmful compliance, sycophantic capitulation, factual correction or another outcome? Treat each as its own measure.
- Model changes: Would the result persist after fine-tuning, pruning, quantization or a deployment change? The reported findings do not establish persistence across those changes, so they should be retested when the evaluated model changes.
What teams can reasonably do with the method
Perturbation probing can be considered a pre-deployment diagnostic for teams able to examine and intervene on a model. It can help investigate whether a targeted behavior appears concentrated in an internal circuit and whether a proposed intervention changes that behavior. It should complement, not replace, evaluation of observable model behavior.
Unit 42 describes the approach as using “only two forward passes per prompt” to identify FFN neurons involved in a targeted behavior, while also recommending defense in depth with external content filters and runtime guardrails. That recommendation is the publisher’s deployment advice, not evidence that any single control—or this diagnostic—establishes safety. In practice, teams should evaluate both model-internal behavior and the deployed system’s responses, and recheck relevant results after material model or deployment changes. Unit 42’s explanatory article provides its account of the approach.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
