A fine-tuned, open-source ticket router reached 90.5% hierarchical intent accuracy on a fixed 200-ticket sample from BANKING77’s test split; across the complete 3,080-ticket test split, it reached 89.8%. The 10,003 tickets were training data, not the evaluation set. These are results reported by the project authors in 2026, not an independent replication.
What the model does
The laya-triage project routes customer-support tickets using a coarse-to-fine hierarchy built on the Laya System 1 decision engine. Instead of choosing directly from 77 possible intents, it first selects one of 12 clusters, then chooses among 3–10 intents within the selected cluster. The project also maps the intents to eight departments.
As an Amazon Associate I earn from qualifying purchases.
A confidence gate can send uncertain decisions to a human and record the reason. The published 421-million-parameter checkpoint is designed for local inference after download, so that inference path does not require an API call or incur a per-ticket inference charge. That describes the model’s local use, not the cost of acquiring or operating the hardware.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →What the 51% and 90.5% figures measure
The 10,003 figure is the BANKING77 training split used to fine-tune routing. Evaluation used tickets from a held-out test split. In the project’s fixed 200-ticket sample from that test split, the zero-shot hierarchical model scored 51.0% accuracy; the fine-tuned hierarchical model scored 90.5%. The project also reports 36.5% for direct, flat 77-way classification on the same sample.
#1 Best Overall
The broader result is 89.8% accuracy across all 3,080 BANKING77 test tickets for the fine-tuned model—not 90.5%. Across that full split, zero-shot hierarchical routing scored 51.1%, while flat classification scored 37.4%. The fine-tuned model’s macro-F1 was 0.897 on the full split and 0.848 on the 200-ticket sample. For the 200-ticket sample, the reported macro-F1 values were 0.443 for zero-shot hierarchical routing and 0.284 for flat classification.
| Configuration | 200-ticket sample accuracy | 200-ticket sample macro-F1 | Full 3,080-ticket test accuracy | Full test macro-F1 |
|---|---|---|---|---|
| Flat 77-way classification | 36.5% | 0.284 | 37.4% | |
| Zero-shot hierarchical routing | 51.0% | 0.443 | 51.1% | |
| Fine-tuned hierarchical routing | 90.5% | 0.848 | 89.8% | 0.897 |
All figures in the table are author-reported BANKING77 evaluation results from 2026. The project used the same 200-ticket sample across the configurations. On the full test split, its paired comparison found a 13.7-percentage-point accuracy gain for hierarchical routing over the flat baseline, with an approximate 95% interval of 11.4 to 16.0 points. The reported breakdown attributes hierarchy’s gains both to better choices among similar intents in a cluster and to recovering tickets that flat routing sent to the wrong cluster.
Rank #2
How confidence-based escalation changes the result
Accuracy alone does not say how many tickets a system handles without review. On the 200-ticket zero-shot sample, the project reports that a confidence threshold around 0.84 auto-handled 60.5% of tickets at 75.2% accuracy; the rest were escalated to a human with a recorded reason. The report cautions that one fewer correct auto-handled ticket would put this observed result below the 75% target.
The fine-tuned model’s reported 90.5% accuracy at 100% coverage on the sample uses a threshold of 0.00. It should be read as a result on this English banking-domain sample, not evidence that a live queue can safely disable escalation. A deployment decision should consider error consequences alongside accuracy, and should report its threshold, auto-handled coverage, escalation rate, and human-review process.
What was fine-tuned—and what was not
The training procedure represents each of the 10,003 training tickets with two routing choices: a coarse cluster and a fine intent, for 20,006 training sequences. It fine-tunes those routing choices. The model’s urgency, frustration, churn-risk, and refund-request signal heads are inherited from the base model; BANKING77 does not supply labels for those signals. The project measured signal outputs separately on a hand-labeled subset and reports substantially different results across languages, so those signals should not be treated as validated outcomes of the BANKING77 routing evaluation.
What the project results establish—and where they stop
The checkpoint is intended for English, banking-style customer-support routing. The project warns that tickets from other domains may be mapped to the nearest banking concept and that the fine-tuned checkpoint does not resolve multilingual quality limitations. Its small multilingual and hand-labeled evaluations do not establish reliable multilingual deployment.
Rank #4
- Built for Local AI Development: AMD Ryzen AI Halo is designed for local AI development and inference, featuring 128GB unified memory and support for up to 200B parameter models to build and run intensive AI workloads locally.
- 128GB Unified Memory: Features 128GB LPDDR5x unified memory at 8000 MT/s with 256 GB/s memory bandwidth, providing a shared memory pool across the CPU, GPU, and NPU to support larger AI models.
- AMD Ryzen AI Max+ 395 Processor: Features 16 cores, 32 threads, and Zen 5 architecture, paired with AMD Radeon 8060S integrated graphics featuring 40 RDNA 3.5 compute units and an AMD XDNA 2 NPU with up to 50 TOPS.
- Linux AI Developer Platform: Purpose-built for Linux-based AI development with full AMD ROCm software support and preloaded tools, models, and workflows optimized for local AI development.
- Compact, Connected Design: Includes a 2TB M.2 SSD, 10GbE LAN, Wi-Fi 7, Bluetooth 5.4, USB-C connectivity, and HDMI 2.1b.
In an out-of-domain IT-operations smoke test, seven of eight tickets were escalated. The remaining SSO lockout was assigned to a banking identity-verification intent. That small test is useful as an illustration of the risk of nearest-concept routing, not as a general measure of out-of-domain performance.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →The project reports 16.1 seconds for a two-pass evaluation of 200 fine-tuned tickets on one Kaggle T4. That is a development measurement, not a production latency benchmark. The documented fine-tuning run used two Tesla T4 GPUs and took about 46 minutes. The project says deployment-target p50/p95 latency and cost per 1,000 tickets remain unmeasured; it also has not run a GPT-4o-mini baseline. Accordingly, these results do not establish production cost, deployment speed, or superiority to an external model.
Best Value
How to inspect or reproduce the work
The project repository links to the evaluation artifacts, results report, Kaggle fine-tuning notebook, and model card. It documents a reproduction command for evaluation. The repository and checkpoint are identified as Apache License 2.0. The authors publish the workflow and its artifacts, but the reported numbers remain project results rather than an independent reproduction.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
