AI changes trust in two directions: it creates new security risks in the systems organizations build or adopt, and it can give both attackers and defenders new capabilities. Security teams should treat AI as part of their security boundary, manage it across its lifecycle, and use layered checks—not a detector score alone—to assess suspicious media.
Why AI changes the security picture
Trust in an AI system depends partly on whether it is secure. The National Institute of Standards and Technology (NIST) identifies “Secure and Resilient” as a primary characteristic of trustworthy AI and notes that AI systems face familiar confidentiality, integrity, and availability concerns involving systems and data. AI also creates risks specific to models and their data, and the security field and its mitigations are evolving rapidly. NIST’s AI security overview also points to AI’s potential to transform cybersecurity; that potential does not mean every attack becomes more effective or that defensive tools are dependable in every setting.
As an Amazon Associate I earn from qualifying purchases.
The practical implication is to assess AI alongside the rest of the software and data environment, rather than treating a model as a standalone feature. The relevant boundary can include the model, its training and output data, configuration, and the software and hardware beneath it. A weakness in any part may affect the confidentiality, integrity, or availability of the system or the services that rely on it.
Free tools Windows power users keep installed
One-click scans. No signup required.
Build and acquire AI systems with lifecycle security in mind
Security work should start before deployment and continue through operation. NIST’s SP 800-218A, published July 26, 2024, extends the Secure Software Development Framework (SSDF) version 1.1 with practices for generative AI and dual-use foundation models. It is intended for model developers, AI system producers, and acquirers, so it is relevant whether an organization builds a model or purchases a system that incorporates one.
#1 Best Overall
For externally developed AI, joint agency guidance recommends secure deployment and operation, with attention to protecting, detecting, and responding to malicious activity affecting the system, its data, and its services. The guidance also emphasizes vulnerabilities and the familiar security goals of confidentiality, integrity, and availability. Read the joint guidance as an operational complement to secure development practices, not a substitute for assessing the particular system and its use.
Put the AI boundary into the security review
- Inventory the model, data, configurations, dependencies, and underlying infrastructure that are in scope.
- Assess conventional software and data risks together with AI-specific risks; do not assume a model’s output or behavior is inherently trustworthy.
- Apply secure development practices when building systems and when evaluating what to acquire, using NIST SP 800-218A as the relevant generative-AI and foundation-model profile.
- For externally developed systems, plan how the organization will protect, detect, and respond to malicious activity affecting the AI system and connected services.
Use a shared language for AI attacks
Teams need a consistent way to describe what an attacker is trying to do, what capabilities are involved, where in the AI lifecycle an attack occurs, and what mitigations may apply. NIST’s AI 100-2 E2025, published in March 2025, provides a taxonomy and terminology for adversarial machine-learning attacks and mitigations. A shared vocabulary can make risk assessments and communication more precise; it does not itself establish that a system is secure or that a particular mitigation will work.
How to assess suspicious or synthetic media
Convincing generated or manipulated media can complicate decisions about what to believe, share, or act on. There is no single check that establishes trust in every case. NIST’s AI 100-4, published November 20, 2024, surveys approaches including provenance tracking, labeling such as watermarking, detection, testing, auditing, and maintenance. These approaches address different parts of the problem and can complement one another.
| Approach | What it is intended to do | What to consider |
|---|---|---|
| Provenance and authentication | Help establish where content came from or whether it has changed. | Consider whether the relevant source information or original file is available. Authentication has limitations and is not a universal guarantee. |
| Labels and watermarks | Signal that content is synthetic or help trace edits. | Consider what the label or watermark can establish in the particular workflow; it should not be treated as proof that all content without one is authentic. |
| Detection | Attempt to identify manipulated media. | Performance may decline when media conditions or generation methods differ from the detector’s training data. A score is not proof that content is real or fake. |
| Response and distribution controls | Address what happens after content is flagged as suspicious. | Plan how reports are assessed and what action is appropriate. Detection alone may not prevent disinformation from spreading. |
This comparison reflects the approaches surveyed by NIST AI 100-4 and the limitations described in the U.S. Government Accountability Office’s 2024 Science & Tech Spotlight on combating deepfakes. The right combination depends on the content and the decision being made; no method should be presented as a universal guarantee.
Rank #3
Set rules for decisions, not just detection
When suspicious media could influence a consequential decision, treat a detector result as one signal among several, not as a verdict. A false positive can cast doubt on authentic material; a false negative can leave manipulated material unflagged. A workflow should therefore define who reviews a flag, what other evidence they consider, and what action is proportionate before content is shared or used to trigger a decision.
Quick Recap
Best Value
- Use provenance or authentication information when it is available, while accounting for its limits.
- Consider labels or watermarks as signals, not as conclusive proof of authenticity or manipulation.
- Use detection as part of a layered process, and account for changes in media conditions and generation methods.
- Connect findings to an organizational response process, since identifying suspicious content does not by itself stop its distribution.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
