Recommended Free Tools
Short answer: Treat an AI purchase as two linked assessments: whether the model is fit for the proposed financial activity and whether the supplier can operate it safely, legally and reliably. Your institution keeps responsibility for outsourced work. Define the use and its consequences, scale diligence to risk, obtain and test evidence, contract for oversight, and monitor the relationship through exit.
Why third-party AI assessment is an institutional responsibility
Outsourcing a model or AI-enabled workflow does not outsource accountability. The Federal Reserve, FDIC and OCC state that a bank’s use of a third party does not diminish its responsibility to operate safely and comply with applicable laws. Their Interagency Guidance on Third-Party Relationships also warns that prior familiarity with a supplier is not a substitute for activity-specific diligence.
As an Amazon Associate I earn from qualifying purchases.
That principle changes the question from “Is this vendor reputable?” to “Can this particular service perform this particular function for these customers, under our controls, with evidence we can review?” A general-purpose language model used to draft internal notes presents a different risk from a model that recommends credit decisions, prices insurance, screens transactions, communicates with customers or initiates payments.
Free tools Windows power users keep installed
One-click scans. No signup required.
The assessment below is a decision process, not a universal certification. Document why the service is appropriate, what remains uncertain and how controls reduce the residual risk.
1. Define the use before evaluating a supplier
Write a short use-case record before sending a questionnaire to vendors. The record becomes the reference point for every later request, test and contract term.
Describe the activity and decision impact
- State the business purpose and the exact workflow the AI will support.
- Identify decisions the output may influence, including approvals, denials, pricing, fraud actions, collections, service access or regulatory reporting.
- List users, affected customers and any vulnerable or protected populations.
- Classify the data involved: public, internal, confidential, personal, financial, authentication, transaction or regulated records.
- Map integrations, tool calls, permissions and whether the system can write to a production record or trigger an external action.
- Specify human review: who reviews outputs, what they must check, and when the AI is prohibited from making the final decision.
- Define expected benefits and measurable acceptance criteria rather than relying on a vendor’s general accuracy claim.
Estimate consequences and scale diligence
Ask what happens if the output is wrong, unavailable, manipulated, biased or disclosed. Consider customer harm, financial loss, legal exposure, safety, market impact, reporting errors and recovery time. A model that only summarizes publicly available material may need a lighter review than an agent connected to payment or lending systems. The interagency guidance calls for treatment proportionate to the activity’s risk and complexity; a copied checklist is not a substitute for that judgment.
Record whether the relationship supports a critical activity, creates material concentration or would be difficult to replace. These findings determine how much evidence, testing, contract protection and monitoring are warranted.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
2. Set an evidence plan matched to the risk
Turn the use-case record into an evidence plan. For each risk, name the evidence you need, who will review it, how often it must be refreshed and what would cause escalation. High-impact uses generally require direct testing, independent review and stronger access rights; low-impact uses may rely more on documentation and sample checks.
Do not treat a SOC report, certification, reference customer or polished demonstration as proof that the service is suitable. Each item is evidence to examine for scope, period, exceptions and relevance to your implementation. The third-party guidance recommends assessing a supplier’s ability to perform, comply and operate safely, with depth scaled to risk and complexity.
Rank #2
3. Assess the vendor’s organization and operating capacity
Model quality cannot compensate for a supplier that cannot protect data, maintain service or cooperate with oversight. Review the organization as carefully as the model.
Ownership, authority and compliance history
- Confirm the legal entity, ownership structure, jurisdictions, licensing and authority to provide the service.
- Review material enforcement actions, litigation, complaints, regulatory findings and unresolved audit issues relevant to the service.
- Map which entity signs the contract and which entities actually process data or operate infrastructure.
- Ask how the vendor tracks laws, regulatory expectations and policy commitments that apply to your use.
Governance, people and segregation of duties
- Identify accountable executives, model owners, security leaders and incident contacts.
- Review staffing levels, turnover in key roles, required expertise and succession plans.
- Check whether development, deployment, approval, security and audit responsibilities are separated where appropriate.
- Understand internal risk committees, change approval, issue tracking and independent testing.
Financial and operational resilience
- Assess financial condition, funding, insurance and the ability to sustain the service through a disruption or business change.
- Review capacity planning, service-level objectives, outage history, recovery time and recovery point targets.
- Examine business-continuity and disaster-recovery tests, including results and open corrective actions.
- Determine whether a failure at the supplier or a key subcontractor would create concentration risk.
Security, privacy and data handling
- Trace data from collection through inference, logging, support access, backup, retention and deletion.
- Verify encryption, identity controls, privileged-access management, vulnerability management, secure development and monitoring.
- Ask whether customer data is used to train or improve shared models, and obtain the contractual setting that governs that use.
- Check data-residency, cross-border transfer, confidentiality, deletion and legal-process procedures for your jurisdictions.
- Review penetration tests, independent assessments, security incidents and remediation evidence for the services in scope.
Subcontractors and technology dependencies
Obtain a current list of material subcontractors, hosting providers, model providers and critical open-source components. For each, identify location, function, access to your data, resilience and notification obligations. Require advance notice or approval for changes that could alter risk. A vendor that cannot explain its dependency chain cannot give you a reliable view of operational or concentration risk.
For credit unions, the National Credit Union Administration’s AI resource page specifically points to due diligence on how an AI product works, the risks it introduces, how it fits the credit union’s business model, and the safeguards, reliability measures and controls in place.
4. Assess the model and the evidence behind it
Understand the model’s intended behavior
Request technical documentation that explains the model or service architecture at a level sufficient for risk assessment: intended purpose, inputs, outputs, operating conditions, prohibited uses, dependencies, update methods and known limitations. Ask whether your implementation uses a foundation model, fine-tuned model, retrieval system, rules engine, agent or a combination. Identify where deterministic controls end and probabilistic behavior begins.
Review data and performance
- Ask what development, fine-tuning, retrieval and evaluation data were used, including provenance, licensing and quality controls where disclosure is possible.
- Request evaluation results for tasks that match your use, not only aggregate benchmark scores.
- Examine performance by relevant customer segment, language, geography, product, operating condition and failure type.
- Ask how hallucinations, unsafe content, prompt injection, data leakage, drift and adversarial behavior are measured.
- Obtain known limitations, incident history, material model changes and the date each result was produced.
Apply model-risk principles where they fit
The Federal Reserve’s revised Supervisory Guidance on Model Risk Management, dated April 17, 2026, emphasizes conceptual soundness, development data, performance, validation, outcome analysis and ongoing monitoring for models within its scope. It also recognizes that proprietary components can make validation harder but do not remove model-risk-management principles. Adapt those practices to the actual technology and use rather than claiming that every AI system is covered by the guidance.
Rank #3
Your validation plan should state assumptions, intended population, operating conditions, institution-specific customization and acceptance thresholds. Test representative cases, edge cases and harmful outcomes. Where the vendor’s test environment differs from production, explain the gap and add local testing.
Do not overstate the 2026 guidance
The revised interagency model-risk guidance is supervisory guidance, not an enforceable or prescriptive standard. It is tailored to model risk, institution size and complexity, and is expected to be most relevant to banking organizations above $30 billion in assets, while potentially informing smaller organizations with significant model risk. The OCC states: “Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance.” Read the OCC Bulletin 2026-13 alongside the Federal Reserve guidance. For generative or agentic systems outside that scope, apply broader governance, third-party, security, privacy and consumer-protection practices.
5. Handle proprietary limits and missing information
Some suppliers will not disclose weights, training data or internal security details. That does not automatically disqualify them, but it changes the confidence you can place in the service.
- Name the missing item. Record exactly what was requested, what was withheld and the reason given.
- Explain why it matters. Link the gap to a specific risk, such as inability to reproduce a result, test bias or verify deletion.
- Seek alternative evidence. Use independent test results, controlled access, red-team reports, audit work papers, demonstrations, samples or contractual attestations where they genuinely address the risk.
- Add compensating controls. Restrict data, narrow permissions, require human approval, increase sampling, isolate the system or prohibit high-impact decisions.
- Set a decision threshold. Accept, conditionally approve, defer pending evidence or select another provider. Document residual uncertainty and the approving authority.
Marketing claims, generic accuracy percentages and brand reputation are not validation evidence. The interagency guidance says prior experience alone is not an adequate proxy for activity-specific diligence and recommends documenting limitations, mitigation and the possibility of using a different provider.
6. Contract for oversight, not just uptime
Contract terms should make the controls you relied on enforceable and observable. Tailor detail to the activity’s risk and complexity, and have qualified counsel confirm jurisdiction-specific requirements.
- Service definition: covered models, versions, environments, permitted uses, performance measures and support commitments.
- Data controls: ownership, permitted processing, training restrictions, retention, deletion, return, location, access and confidentiality.
- Security: control standards, testing, vulnerability remediation, access management, logging and cooperation with investigations.
- Incident response: notification deadlines, required content, evidence preservation, customer and regulator cooperation, and remediation duties.
- Model and service changes: notice, impact information, testing windows, approval rights, rollback and emergency-change procedures.
- Subcontractors: disclosure, flow-down obligations, material-change notice and rights to object or terminate.
- Audit and information access: access to relevant records, independent reports, testing results and personnel, subject to appropriate confidentiality controls.
- Regulatory access: provisions supporting examination and access to documentation and materials where applicable, as discussed in the interagency guidance.
- Continuity and exit: recovery commitments, data export formats, transition assistance, deletion certification, intellectual-property treatment and cooperation with an orderly transfer.
7. Monitor the relationship throughout its lifecycle
Approval is the start of oversight. Establish a monitoring schedule based on risk, then increase it when performance, threat conditions or the business use changes.
Monitor service and model performance
- Track accuracy, reliability, latency, availability, abstention and harmful-output rates against approved thresholds.
- Sample production outcomes and compare them with validation results and customer-impact indicators.
- Review model, prompt, retrieval, tool, data and infrastructure changes before or soon after release.
- Watch for drift in data, customer populations, languages, products and operating conditions.
Monitor the vendor and control environment
- Refresh financial, security, compliance, insurance, audit and subcontractor information at defined intervals.
- Track incidents, outages, material findings, overdue remediation and changes in key personnel.
- Verify that certifications and SOC reports still cover the service, period, locations and controls you rely on.
- Test incident-notification and escalation contacts instead of assuming they work.
Prepare escalation and exit
Define triggers for suspension, restricted use, remediation, executive escalation or termination. Triggers may include material audit findings, financial deterioration, security incidents, data loss, repeated service interruption, compliance failures or unapproved subcontractor and model changes. Maintain an exit plan covering data, credentials, integrations, intellectual property, customer communications, records and replacement capacity. The Financial Stability Board’s 2023 third-party risk toolkit emphasizes lifecycle management, critical services and systemic dependency risks; it is an international, flexible toolkit rather than a single global rulebook.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.8. Compare viable providers on the same decision axes
Once unsuitable providers are removed, compare the remaining options using a documented scoring method approved for your institution. The agencies do not prescribe a universal rating scale, so set thresholds that reflect your use and risk appetite.
| Decision axis | Questions to answer | Evidence to retain |
|---|---|---|
| Intended-use fit | Does the service perform the defined task under our data, workflow and human-review conditions? | Use-case tests, limitations, acceptance results |
| Validation transparency | Can we obtain relevant evaluation results, segment performance, change history and test access? | Technical documents, test reports, independent assessments |
| Privacy and security | Are collection, training, retention, access, encryption and incident controls adequate? | Data-flow map, contracts, audit reports, security findings |
| Legal and compliance support | Can the supplier support applicable obligations and regulatory inquiries? | Entity information, compliance history, response procedures |
| Financial and staffing capacity | Can it sustain the service and retain critical expertise? | Financial review, staffing plan, insurance, continuity tests |
| Resilience and recovery | What happens during an outage, cyberattack or subcontractor failure? | Recovery objectives, test results, incident records |
| Concentration and dependencies | Would shared infrastructure or subcontractors create correlated failure? | Dependency map, provider concentration analysis |
| Change and incident notice | Will we know about material model, service, personnel or control changes quickly enough to act? | Notification terms, release process, escalation contacts |
| Audit and monitoring rights | Can we verify controls and investigate problems throughout the contract? | Audit clauses, report scope, access procedures |
| Portability and exit | Can data, configurations and customers be moved without unacceptable harm? | Export specification, transition plan, deletion process |
Record trade-offs rather than hiding them in a single score. A provider with stronger technical performance but poor evidence access may be less suitable for a high-impact use than a slightly less capable provider that supports independent validation and reliable exit.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →9. Keep jurisdiction and institution type in view
U.S. banks should read the 2023 interagency third-party guidance and the April 17, 2026 model-risk guidance together, while recognizing their different purposes and scope. The model-risk document is not a blanket rule for every generative or agentic system. Smaller institutions can still use its concepts when their model risk warrants it.
Credit unions should consult NCUA’s AI materials, including its provider-diligence topics, and align the process with their own supervisory obligations. Institutions operating internationally can use the FSB toolkit to identify critical third-party services and lifecycle or systemic dependencies, then map those practices to local law and supervisory expectations. None of these sources is a universal certification or a replacement for legal advice in each jurisdiction.
10. A practical approval record
Your final assessment file should let an independent reviewer reconstruct the decision without relying on verbal assurances. Include:
- the approved use, users, affected customers, data classes, integrations and prohibited uses;
- risk and criticality ratings with rationale;
- vendor ownership, financial, staffing, security, compliance, subcontractor and resilience evidence;
- model documentation, evaluation results, local validation, limitations and unresolved assumptions;
- information gaps, compensating controls, residual risk and approval authority;
- contract obligations, audit rights, notification terms, change controls and exit provisions;
- monitoring metrics, review frequency, escalation triggers and named owners;
- the next review date and conditions that require reassessment.
Common assessment failures and fixes
“We already know the vendor”
Replace relationship-based comfort with evidence tied to the exact service, data and decision impact. A familiar supplier can introduce new risk through a different model, subcontractor or integration.
“The certificate proves compliance”
Check the report’s scope, period, exceptions, complementary user controls and relevance to your deployment. Add testing for risks the report does not cover.
“The model is proprietary, so validation is impossible”
Request alternative evidence, conduct black-box and outcome testing, restrict use and document the remaining uncertainty. If the residual risk exceeds tolerance, choose another service.
“Approval ends the work”
Set monitoring, change-notice, incident and exit obligations before production. Reassess when the model, data, customer population, regulation or dependency chain changes.
“The 2026 guidance covers every AI system”
It does not. The revised guidance excludes generative and agentic AI and is supervisory rather than prescriptive. Apply broader governance and third-party controls to systems outside its scope.
Decision rule
Approve a third-party AI service only when the intended use is clearly bounded, the vendor can demonstrate operational and control capacity, model evidence is sufficient for the consequences involved, information gaps have explicit mitigations, contractual oversight is enforceable, and monitoring and exit are feasible. If any of those conditions cannot be met, narrow the use, add controls, defer deployment or select a different provider.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
