A ternary neural network uses three possible values—most often −1, 0, and +1—for selected parts of the model, usually its weights. The three-state representation can reduce weight storage and replace some multiplications with signed additions or omitted terms, but the term alone does not tell you whether activations are also quantized or whether inference will be faster on a particular device.
What does “ternary neural network” mean?
“Ternary” describes the number of allowed states: three. In the common case, each quantized weight is negative, zero, or positive, represented as −1, 0, or +1. It does not specify which model tensors use those states. A paper may describe ternary weights, ternary activations, or both, so check what it quantizes before comparing methods.
As an Amazon Associate I earn from qualifying purchases.
In a full-precision network, weights can take many floating-point values. A binary-weight network typically restricts them to −1 and +1. Ternary weights add zero, allowing a connection to contribute nothing to a calculation. That zero state can also make the weight matrix sparse.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsHow do ternary weights work in a neural network?
During a dot product, a full-precision weight generally requires a multiplication. With ternary weights, a value of +1 passes the corresponding input through, −1 negates it, and 0 allows the term to be skipped. This is why ternary designs are intended to reduce arithmetic work. Whether a device can exploit the representation efficiently depends on its hardware and implementation.
#1 Best Overall
The three labels do not always mean the deployed nonzero weights have equal magnitude. Methods may multiply ternary values by a scale, and some learn separate positive and negative scales. For example, Ternary Weight Networks approximate weights using ternary values and a scaling factor, while Trained Ternary Quantization learns separate positive and negative scale coefficients. These are different training choices, not a single universal rule.
How are ternary networks trained?
Training must determine which values become negative, zero, or positive, and how the nonzero values are scaled. Approaches differ in how they choose thresholds, optimize the quantizer alongside the weights, and control the number of weights assigned zero. Sparsity-control methods explicitly address the zero fraction; other methods use different quantization or scaling strategies.
Rank #2
Consequently, “ternary” does not guarantee identical deployed levels, sparsity, or accuracy across models. For example, a method with learned positive and negative scales can have three states whose magnitudes are asymmetric. The training method and final representation matter as much as the three-state label.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →What are the potential benefits and trade-offs?
- Less weight storage: Three states have an ideal information content of log₂(3), or about 1.58 bits per value. That is not automatically the file size per parameter: a straightforward fixed-width encoding takes two bits per value, and scales, metadata, and other model components add storage.
- Less multiplication work: Multiplication by −1, 0, or +1 can be handled as negation, omission, or pass-through rather than general multiplication. The practical benefit depends on whether the hardware kernel uses those operations efficiently.
- Potential sparsity: Zero-valued weights can reduce the number of active connections, but the actual sparsity depends on training and the quantizer.
- Implementation and accuracy trade-offs: Quantization choices, scale overhead, activation precision, and hardware all affect the result. A smaller representation does not by itself establish accuracy parity or lower end-to-end latency.
FATNN, presented at ICCV 2021, discusses the distinction between ideal ternary information content and two-bit fixed-width encoding, as well as a particular acceleration method. Those implementation results should not be read as a speed guarantee for other models or processors.
Rank #3
How should two ternary-network methods be compared?
A useful comparison needs matched conditions rather than the shared word “ternary.” Check:
- Whether weights, activations, or both are quantized.
- The three deployed levels and whether positive and negative weights use different scales.
- Accuracy against the same full-precision baseline, on the same task and dataset.
- Effective model storage, including scales and metadata rather than just nominal bits per weight.
- Latency or energy measured on the same hardware, workload, and implementation conditions.
- The fraction of weights set to zero and whether the hardware exploits that sparsity.
Without those details, a claim that one ternary method is smaller, faster, or more accurate than another may not be an apples-to-apples comparison.
Quick Recap
Best Value
Further reading
- Ternary Weight Networks (2016) introduces ternary weight approximation with a scaling factor.
- Trained Ternary Quantization (ICLR 2017) describes learned positive and negative scaling coefficients.
- Sparsity-Control Ternary Weight Networks (2020) focuses on controlling zero-valued weights.
- FATNN: Fast and Accurate Ternary Neural Networks (ICCV 2021) discusses ternary encoding and a specific acceleration approach.
- Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network Using Truncated Gaussian Approximation (CVPR 2019) studies joint weight and quantizer optimization.
- Ternary Neural Networks for Resource-Efficient AI Applications (2016) is another early treatment of ternary networks.
- TRQ: Ternary Neural Networks With Residual Quantization (AAAI 2021) explores residual quantization.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
