You can assess AI risk without making assumptions about superintelligence. Start with the system as it will actually be used: what it does, who relies on it, who could be affected, and what happens if it fails. Then examine relevant trustworthiness dimensions, gather evidence through appropriate tests, and revise the assessment as the system and its setting change.
Why AI risk depends on context
There is no single risk level that applies to “AI” as a whole. Risk depends on the system, its task, where and how it is deployed, and the people who may rely on or be affected by it. A model evaluated in isolation may behave differently when embedded in a product or a larger workflow.
NIST’s voluntary AI Risk Management Framework (AI RMF) is designed to help organizations manage risks to individuals, organizations, and society. NIST released AI RMF 1.0 on January 26, 2023; its status can change, and NIST says the framework is being revised. Check the NIST AI RMF page for current information. The framework supports risk management; it is not a certification or guarantee that a system is trustworthy.
A practical sequence for evaluating risk
1. Define what you are evaluating
Set the unit of analysis before testing. You might assess a model, a finished product, or an entire deployed workflow. Describe its capabilities and components, intended users and uses, and boundaries: what it is not designed or authorized to do. Include relevant versions and dependencies so that findings can be tied to the system that was actually assessed.
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
2. Map the deployment and the people involved
Consider the setting in which the system operates, not only its technical design. Ask:
- Who uses the system, and who may be affected without using it directly?
- What decisions or actions does it influence, and how consequential are they?
- What could happen if it gives an incorrect, incomplete, or misleading output, or becomes unavailable?
- Can a person review, override, or appeal its output? Who is responsible for doing so?
- Do the actual users, data, and operating conditions match the assumptions made during development?
These questions help make contextual risks visible; they are practical prompts, not a verbatim NIST checklist.
3. Identify risks across relevant dimensions
Use more than accuracy as your lens. NIST’s trustworthiness guidance covers validity and reliability, safety, security and resilience, accountability and transparency, explainability, privacy, and harmful bias. Which dimensions matter most depends on the task and setting. For example, a system that handles sensitive information raises privacy and security questions, while one that informs decisions about people may require close attention to harmful bias, accountability, and review.
Do not collapse the assessment into a single score and treat that as the answer. A score can obscure distinct weaknesses, such as strong average performance alongside poor results for a particular group, or reliable outputs in testing alongside unsafe behavior in the actual workflow. NIST also cautions that considering trustworthiness characteristics cannot by itself ensure trustworthiness. See the NIST AI RMF FAQ.
Rank #3
4. Match evidence to the risk
Choose tests that address the system and the conditions that matter. NIST’s Assessing Risks and Impacts of AI (ARIA) program describes model testing, red-teaming, and field testing. These approaches provide different kinds of evidence:
- Model testing: controlled evaluations of model performance on specified tasks and conditions.
- Red-teaming: adversarial exercises that probe for weaknesses, including ways the system might be misused or behave unexpectedly.
- Field testing: evaluation in a real or representative use context, where workflows and human interactions can affect outcomes.
ARIA emphasizes technical and contextual robustness as well as performance and accuracy. A benchmark result therefore answers a bounded question about the tested conditions; it does not prove that a system is safe in every deployment. When recording results, state what was tested, how, under which conditions, and how closely those conditions resemble real use. Note limitations and any important populations or situations that were not covered. See NIST ARIA.
Rank #4
5. Monitor changes and learn from incidents
An assessment can become outdated when the model, data, users, workflow, or operating environment changes. Keep a record of material changes and observed failures or impacts, then use that information to revisit risks, safeguards, and tests. Incident reporting is useful for learning and comparison; it is not, on its own, proof of how common AI harms are.
In 2025, the OECD published a common framework with 29 criteria for capturing and comparing AI incidents across contexts. Those 29 criteria describe a reporting structure, not 29 incidents, an incident count, or a risk rate. See the OECD framework.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteApplying the approach to generative AI
Generative AI adds risks that may arise from producing or transforming content, but the same basic sequence applies: define the system and use, map the context, identify relevant harms, test under appropriate conditions, and monitor what happens in deployment. NIST’s Generative AI Profile was released on July 26, 2024 to help organizations identify generative-AI-specific risks and consider management actions aligned with their goals. It complements the AI RMF rather than replacing context-specific evaluation. See the NIST Generative AI Profile.
For practical resources to operationalize risk work, NIST’s AI Resource Center provides materials related to testing, evaluation, verification, and validation.
Keep conclusions proportionate to the evidence
A useful assessment explains what system and use were examined, which risks were considered, what the tests showed, and what remains uncertain. It also records mitigations and who is responsible for monitoring them. Passing a test or applying a framework is evidence of work completed, not a promise that no harm will occur.
This process addresses present systems and their real uses. It does not resolve speculative questions about future superintelligence; those questions are distinct from assessing the reliability, safety, security, privacy, fairness, and impacts of systems people use today.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
