Free tools Windows power users keep installed
One-click scans. No signup required.
To benchmark C++ assignment solvers fairly, first define exactly which assignments are legal and what the solver must optimize. Then test a documented mix of square and rectangular, dense and sparse workloads; validate every answer; and publish timings with enough environment and input detail to reproduce them. A matrix of random square costs alone cannot establish performance on a real placement workload.
Define the assignment problem before comparing solvers
“Assignment solver” can refer to implementations with different rules. Two runtimes are comparable only if both programs solve the same mathematical problem and return results under equivalent input handling.
As an Amazon Associate I earn from qualifying purchases.
Document the contract for every benchmark suite:
- Shape and cardinality: Are matrices square or rectangular? Must every item on the smaller side be matched, or is a different number of assignments allowed?
- Unmatched items: Can agents or tasks remain unassigned? If so, what is the objective impact, if any?
- Forbidden pairs: Are missing edges prohibited, or represented by a penalty cost? State how each solver receives that information.
- Objective and numeric representation: Are costs minimized or maximized, and what numeric types and ranges are supported?
- Failure behavior: What should happen when no feasible assignment exists?
These details matter for rectangular matrices in particular. The OR-Tools linear assignment documentation describes assignment as costs between agents and tasks, and its example allows workers to remain unassigned when there are more workers than tasks. A benchmark should not assume that every implementation uses the same rectangular or unmatched-item semantics.
If an implementation requires padding or another transformation, document it and explain how it affects costs, feasibility, and the returned objective. Do not compare outputs until those transformations preserve the same assignment contract.
#1 Best Overall
Build a workload suite that reflects placement
Benchmark across workload strata rather than relying on one matrix family. At minimum, vary dimensions, aspect ratios, allowed-edge density, and cost distributions. Include repeated values or ties when they occur in the intended application.
| Workload dimension | What to vary | Why it matters |
|---|---|---|
| Dimensions and shape | Multiple sizes; square and rectangular matrices with different aspect ratios | Size and rectangularity affect both the amount of work and the behavior of implementations. |
| Allowed-edge density | Dense and sparse regimes, with a documented definition of density | A solver’s behavior on mostly allowed pairs may differ from its behavior when many pairs are forbidden. |
| Cost values | Ranges and distributions representative of the application; include ties and repeated values when applicable | Unstructured random values may omit patterns present in placement costs. |
| Structural pattern | Documented placement traces or generators tied to observed placement properties | This is the evidence needed to call a benchmark placement-realistic rather than merely synthetic. |
| Difficulty class | Easy, typical, and difficult instances, classified using observed workload properties | Labels are useful only when the underlying properties and classification rule are stated. |
Existing benchmark materials show that matrix size and dense-versus-sparse cases are useful comparison axes, but their timings describe their own implementations and setups. A solver benchmark repository describes testing by matrix size and implementation type; a separate C++ repository reports dense and sparse timing tables, including sparse matrix sizes from 8 through 1024. Neither establishes a standardized placement workload suite or a universal ranking.
No independently validated placement-workload performance figure or standard public placement benchmark suite is established by these sources. If actual placement traces or a documented generator are unavailable, call the suite synthetic or application-inspired, not a validated realistic placement benchmark. Explain the connection between any generator and observed placement data rather than relying on the label.
Validate every result before measuring speed
Correctness checks should run before performance claims. For each result, independently verify feasibility and recompute the objective using the original costs, not a transformed matrix or a value reported only by the solver.
- Every assigned pair is allowed under the benchmark contract.
- No row or column is used more often than permitted.
- The assignment has the required cardinality, including the contract’s treatment of unmatched items.
- The reported objective matches a fresh calculation from the original input costs.
- Infeasible inputs produce the documented failure behavior when the application can encounter them.
For a subset of small instances, compare results against a trusted exact formulation or an independent enumerator. This is a recommended benchmark control, not a validation protocol prescribed by the cited solver documentation. Keep the validator independent enough that it can catch a solver or input-conversion error.
Make timing measurements reproducible
Record the machine, software, inputs, and measurement procedure alongside results. Report elapsed time and memory use when memory affects deployment. Keep input generation and output validation outside the timed region unless the intended measurement is explicitly end-to-end.
- Hardware and system: CPU model, memory, and operating system.
- Build and solver: Compiler and version, optimization flags, solver or library version, and thread count.
- Inputs: Generation method, random seed, dimensions, density, cost properties, and any preprocessing or matrix transformation.
- Measurement policy: Warm-up procedure, number of repetitions, timing statistic, and timeout policy.
- Timed boundary: Whether allocation, conversion, and preprocessing are included or excluded.
Show per-instance results or distributions for each workload stratum as well as any aggregate summary. Explain outliers and timeouts rather than silently dropping them. Scaling plots or tables by dimension and density are more informative than a single overall number because they show which workload classes drive the result.
Published repository timings remain specific to their code and test environments. The dense and sparse timings in the C++ repository’s results should not be recast as performance on realistic placement data or transferred to another machine, implementation version, or compiler configuration.
Best Value
Compare implementations within their actual scope
For pure linear assignment, compare specialized assignment algorithms under the same input contract. Include MIP or CP-SAT formulations when placement rules need their additional modeling flexibility, but separate model-building and solver overhead from the core assignment kernel if the aim is to compare kernels.
OR-Tools distinguishes its specialized linear sum assignment solver from more general MIP and CP-SAT formulations. Its linear assignment documentation presents the specialized solver for simple assignment and the broader formulations for more complex scenarios. That is a scope distinction, not evidence that one approach is always faster for placement.
Algorithm names are not enough to establish equivalent performance. The OR-Tools C++ Hungarian reference describes its documented Kuhn–Munkres implementation as “An O(n^4) implementation of the Kuhn-Munkres algorithm (a.k.a. the Hungarian algorithm) for solving the assignment problem.” The documentation, last updated in 2024, explicitly advises using graph/linear_assignment.h, whose complexity is usually much smaller. This is a complexity statement about that documented implementation, not a measured runtime result or a characterization of every implementation called Hungarian.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsA separate C++ implementation page describes rectangular dimensions and O(rc min(r,c)) complexity while incorporating Jonker–Volgenant ideas. Treat that as a property claimed for that implementation, not as a guarantee for every algorithm or workload.
What a defensible benchmark can conclude
A benchmark can establish how specified, versioned implementations behaved on a documented machine and workload suite. It cannot support a universal winner when the assignment contracts, inputs, code, or hardware differ. If placement realism is claimed, document the trace or generator’s connection to actual placement data; otherwise describe the suite accurately as synthetic or application-inspired.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
