Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Yes—a deduplication pass can erase repetition-based training weights if the pipeline represents a higher weight by copying examples and then removes duplicates. In a postmortem published on DEV Community on August 23, 2026, Seth Wheeler describes exactly that failure in his training-data pipeline: the repeated rows disappeared before training, and a suffix-based deduplication key also merged some examples that were not identical.
What happened in Wheeler’s training pipeline?
Wheeler built a capture-the-flag arena in which language models attacked and defended containers. He extracted game turns to train a local Qwen2.5-3B-Instruct bot with an MLX LoRA adapter, which he describes as rank 8 across 8 layers. The bot performed better on offense than defense, so he tried two curriculum approaches intended to emphasize defensive examples. These are Wheeler’s reported results and analysis of his own code and logs, not independently verified findings. Wheeler’s DEV Community account describes the project.
As an Amazon Associate I earn from qualifying purchases.
The important pipeline detail was how it encoded importance. In Wheeler’s account, training/defense_training_data.py, on the custom-bot branch, assigned weights to turns based on outcomes and vulnerability checks. It implemented a weight by repeating the example in the JSONL list: a weight of 6 meant six copies of that record before later processing.
Free tools Windows power users keep installed
One-click scans. No signup required.
The intended weights
- Successful defensive turns received weight 6.
- Patched defensive turns that still lost a flag received weight 3.
- Neutral defensive activity received weight 1; turns marked to drop received weight 0.
- Attacker commands that captured a flag received weight 3; other attacker turns received weight 1.
The function then deduplicated the examples using line[-200:] as its key. Once it had seen a key, it skipped later records with the same key. Since repeated copies of a given example had the same text, the deduplication step removed the copies that represented the higher weights. As Wheeler put it, the weighting and deduplication stages were “the same operation with opposite signs.”
#1 Best Overall
Why did the deduplication key cause a second problem?
The key compared only the last 200 characters, not the full example. Two lines could therefore have different reasoning earlier in the text but end with the same command and share a key. Wheeler says the pass removed some of these distinct full lines as well as the intended duplicates.
That distinction matters when diagnosing the result. Full-line deduplication would remove repeated identical examples; a suffix-key collision can discard different examples because the chosen fragment is not a unique identifier for the whole record. Whether a shorter key is appropriate depends on what makes an example meaningfully distinct, but the key should match that definition rather than merely being convenient.
Rank #2
- Use scikit-learn to track an example ML project end to end
- Explore several models, including support vector machines, decision trees, random forests, and ensemble methods
- Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection
- Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformers
- Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning
How many examples survived the pass?
Wheeler reports that a run over 358 session directories—329 of them with a complete game.json—produced the following counts. These are figures from his analysis, not a reproduced or independently verified corpus audit.
| Pipeline stage or effect | Examples | What the count represents |
|---|---|---|
| After weighting, before deduplication | 94,022 | Rows including copies used to represent weights |
| Distinct full lines | 55,034 | Unique complete lines before the suffix-key pass |
| After suffix-key deduplication | 50,145 | Rows remaining after the pipeline’s deduplication |
| Weighting repetitions removed | 38,988 | 41.5% of the weighted corpus, according to Wheeler |
| Additional distinct lines removed by suffix collisions | 4,889 | 8.9% of the 55,034 distinct full lines, according to Wheeler |
The counts show why a single “deduplicated” total is not enough to explain what happened. The 38,988 removed repetitions were intended to carry weight; the additional 4,889 removals were lines that differed in full but collided on the 200-character key.
Rank #3
What the intended weighted corpus contained
Wheeler reports this breakdown for the 94,022 rows before deduplication:
| Example group | Source turns or lines | Intended weighted rows | Weight |
|---|---|---|---|
| Successful defensive turns | 2,861 | 17,166 | 6 |
| Patched but lost defensive turns | 555 | 1,665 | 3 |
| Neutral defensive turns | 18,227 | 18,227 | 1 |
| Attacker commands that captured flags | 2,518 | 7,554 | 3 |
| Other attacker turns | 33,035 | 33,035 | 1 |
| Curated Gemini defensive exemplars | 3,275 | 16,375 | 5 |
He says the successful “gold” defensive turns were meant to make up 18.3% of the weighted data, but represented at most 5.7% of the post-dedup training set. The curated Gemini file contained 3,275 lines and 3,187 distinct 200-character tails; Wheeler reports that increasing its configured repetition weight from 2 to 5 therefore did not change how many examples survived deduplication.
Rank #4
What did the trainer actually receive?
Wheeler says the downstream split contained 38,448 training lines and 4,272 validation lines, 42,720 in total. He reports that every training line was unique both as a full line and by its 200-character tail. The source also says session logs, JSONL corpora, and adapters were gitignored, so readers could inspect the extraction and retraining code but not the generated data files.
This is the operational lesson: upstream counters can show that a weight was assigned without proving that the corresponding examples reached the trainer. The meaningful check is the corpus after every transformation, including the final split—not just the scoring logic that initially constructs it.
Best Value
Does the bug explain the bot’s regression?
Not by itself. Wheeler explicitly says he has not run the experiment needed to establish that the lost weighting caused the retrains to be rolled back: use the same data, preserve the weights, and compare under the same evaluation. Correct weighting might still have regressed. He also notes that his extraction run used more sessions than were available at the time of the v7 training run, so the absolute corpus counts cannot be compared directly with that historical run.
His article reports that across 11 self-play generations, validation loss fell from 0.526 at generation 0 to 0.462 at generation 9, with a reported best of 0.377. Over the same span, average score fell from 172.2 to 101.4 and flags lost per game rose from 2.7 to 4.7. Those figures are Wheeler’s report, not an independent benchmark. He cautions that matches among six copies of the same bot are close to zero-sum and do not measure general capability. The result supports a narrower observation: the validation metric improved while the behavior he wanted did not. It does not establish why.
Wheeler also reports pooled results for 221 logged games: his custom bot captured 400 flags and lost 134; GPT-OSS 120B captured 275 and lost 163; GLM-5.1 captured 237 and lost 160; Nemotron 3 Super captured 61 and lost 133; Gemini 3 Flash captured 55 and lost 40; and RNJ-1 8B captured 2 and lost 131. He warns these are not a clean adapter-version comparison: the logs do not record which adapter version played, and the match format changed during the tournament. The numbers should not be read as proof that a particular adapter or curriculum caused a performance difference.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchHow to prevent repetition weights from disappearing
- Define what “duplicate” means. Decide whether two examples are duplicates only when the entire training record matches, or whether a particular field should define identity. Avoid treating a short suffix as a full-record key unless that suffix is guaranteed to distinguish the examples you need to preserve.
- Measure counts and composition at each stage. Record total rows and counts by class or assigned weight before weighting, after weighting, after deduplication, and after splitting. A total alone can reveal a large loss; class-level counts show which examples lost representation.
- Test the exact deduplication key. Include cases with identical records, different reasoning but identical endings, and records that differ only outside the key. Verify which ones should survive, then make the code match that expectation.
- Choose an explicit representation for importance. If examples are repeated to simulate weights, ensure later uniqueness stages do not collapse those copies. Alternatively, keep a weight as data and use a training method that actually consumes it; a stored weight field has no effect unless the trainer applies it.
- Audit the final training input. Inspect the corpus and split that the trainer actually reads. Confirm both the final row count and the distribution of examples that were meant to receive extra emphasis.
- Isolate the causal test. To determine whether lost weights changed model behavior, compare otherwise identical runs with and without the intended weighting, using the same dataset, evaluation procedure, and scoring conditions.
The check that would have exposed Wheeler’s failure early was a before-and-after corpus count. The more complete safeguard is to verify not only how many rows reach training, but also whether the intended distinctions and proportions remain intact.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
