For AI used in financial services, insurance, or other regulated work, a convincing explanation is not enough. Buyers need to know what evidence and rules produced a decision, whether they can reproduce its path later, and who is responsible if it is wrong. Graham French, chief technology officer of UnlikelyAI, says these questions are arising in procurement conversations he has had; they are reported observations, not a representative survey of buyers.
What regulated buyers want to know
French describes buyers asking four connected questions:
As an Amazon Associate I earn from qualifying purchases.
- How did the system reach this output, and what evidence supports its correctness?
- Can the decision’s reasoning be reconstructed months later if someone challenges it?
- Which sources and rules informed the result?
- Who is accountable when the result is wrong?
The key distinction is between an explanation and a record. A language model can generate a fluent account after making a decision, but that account alone does not establish which sources or process actually produced the result. For an auditable decision, the system needs a record of the inputs and decision path—not merely a plausible story about them.
Free tools Windows power users keep installed
One-click scans. No signup required.
How to test a vendor’s answer
Rather than accepting a general claim that a system is explainable, ask the vendor to demonstrate a specific prior consequential decision. French recommends asking them to reproduce it and show the path it followed. This is a practical procurement test, not a formal standard or independently validated benchmark.
#1 Best Overall
- Choose a real case. Use a prior decision that matters to your organisation and can be examined under your data-handling and confidentiality rules.
- Ask for the original inputs. Have the vendor identify the source material used, including the version or date where relevant.
- Trace the decision. Ask which rules or reasoning steps led from those inputs to the result, and what the system recorded at the time.
- Re-run it. See whether the vendor can reproduce the result and explain any difference from the original.
- Challenge the case. Change an input or introduce an edge case. Ask what changes in the output and whether the path remains inspectable.
- Establish responsibility and maintenance. Clarify who investigates an error, who can change decision rules, and how policy updates are reflected in the system.
A useful demonstration should make it possible to distinguish what the system actually used from what it can generate as a post-hoc explanation. The vendor should also be able to describe how its records remain available for later review.
What an auditable system may look like
French presents neurosymbolic AI as one possible design: a language model processes unstructured material, while an explicit rule system makes the decision and produces a traceable path. The proposal is not evidence that this architecture is best for every use case. It is one way to separate flexible input processing from the rules governing a consequential result.
Rank #2
Explicit rules bring trade-offs. Domain experts must write and maintain them as policies change, which can take more time and money than a model-driven approach. In return, a rule-based path can make it easier to reconstruct decisions, test edge cases, and correct a policy without retraining a model. Buyers should evaluate those costs against the consequences of an untraceable or difficult-to-correct decision.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Why governance confidence matters
The AI Journal article attributes several figures to Grant Thornton’s 2026 AI Impact Survey: 78% of senior leaders reportedly lacked strong confidence that they could pass an independent AI governance audit within 90 days, and 46% named governance failures as a leading cause of AI underperformance. It also reports a contrast by deployment stage: 7% of organisations still piloting AI were very confident of passing that audit, compared with 74% of organisations running AI in full production. These are figures as the article reports them; the survey itself has not been independently verified here.
French says explainability and auditability requirements are appearing in procurement documents. That is his reported observation, not a measured trend. For a buyer, the practical implication is to turn broad assurances into requirements that can be demonstrated on real cases: what gets recorded, how a decision can be reconstructed, how edge cases are tested, who owns corrections, and how rule changes are controlled.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How the EU AI Act deadline fits
A Grant Thornton UK legal briefing says standalone Annex III high-risk AI systems have until 2 December 2027 to comply under Regulation (EU) 2026/1744, which it says entered into force on 27 July 2026. This is a secondary legal summary; it does not establish whether a particular system or organisation is in scope. Buyers should assess applicability to their specific use and jurisdiction rather than treating the date as a universal deadline.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →
