October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk6 min

How to Audit AI Agent Activity and Investigate Unwanted Changes

A practical investigation workflow for tracing AI agent activity, identifying the acting identity, confirming resource changes, and documenting evidence limits.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

To investigate an unwanted AI-agent change, build a timestamped evidence chain from the initiating person or service through the agent’s identity, session, model and tool activity, approvals, and the affected system’s own audit record. The agent trace helps reconstruct what was attempted; the target system’s audit trail is the key evidence that a resource operation actually occurred.

What to collect before investigating

Start with the change you observed, not just the agent you suspect. Record the affected resource, environment, when the change was first noticed, and any known actor or session. Then preserve the relevant records before routine processing or retention limits remove context.

As an Amazon Associate I earn from qualifying purchases.

  • Save original trace, identity, approval, network, and target-system audit exports where available.
  • For each export, record the source system, query or filter, time range, export time, and timezone.
  • Keep original timestamps and identifiers. If you normalize times for a shared timeline, retain the originals as well.
  • Preserve the resource’s current state and any change or restoration evidence according to your incident process.

These are collection practices, not a guarantee that any particular vendor retains records for a specific period. Availability depends on configuration, permissions, and the source system’s retention settings.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Reconstruct the event in a reliable order

1. Identify every actor in the chain

Do not collapse the person who initiated the work, the application, agent blueprint, agent instance, and service principal into one “agent” identity. Preserve the exact actor and target values each system records, then identify which identity was authorized to perform the resource operation.

#1 Best Overall
Yubico - Security Key C NFC - Basic Compatibility - Multi-Factor authentication (MFA) Security Key and passkey, Connect via USB-C or NFC, FIDO Certified
  • POWERFUL SECURITY KEY: The Security Key C NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
  • WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key C NFC secures 100 of your favorite accounts, including email, password managers, and more.
  • FAST & CONVENIENT LOGIN: Plug in your Security Key C NFC via USB-C and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
  • TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
  • BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.

For Microsoft Entra Agent ID records, inspect agentType, initiatedBy, performedBy, targetResources, and blueprintId. A blueprint ID can link an agent instance to its blueprint. Agent sign-ins may appear in different sign-in log types depending on whether permissions were delegated or app-only. See Microsoft Entra Agent ID sign-in and audit logs.

2. Rebuild the agent execution

Open the relevant session and follow its turns and spans. Identify the model-generation event, tool name, arguments, returned result when available, outcome status, and any error details. A tool call can be attempted, rejected, or fail; do not treat its presence in a trace as proof that the target resource changed.

OpenAI documents generation spans with recorded inputs and outputs and tool spans with tool name, arguments, result when available, status, and error details. Session trace export is available as OTLP JSON when tracing is enabled and the API key has the appropriate trace or agent read permission. Traces may be available only after turns finish, and recorded content depends on platform behavior and organizational data controls. See OpenAI Agents API tracing.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #2
Yubico - YubiKey 5 NFC - Multi-Factor authentication (MFA) Security Key and passkey, Connect via USB-A or NFC, FIDO Certified - Protect Your Online Accounts
  • POWERFUL SECURITY KEY: The YubiKey 5 NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
  • WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5 NFC secures 100+ of your favorite accounts, including email, password managers, and more
  • FAST & CONVENIENT LOGIN: Plug in your YubiKey 5 NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
  • MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
  • PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts

For Azure SRE Agent, documented customEvents cover model generation, tool execution, session lifecycle, routing, and handoff. Tool telemetry can include the tool name, input, output, calling subagent, and call ID. Shared fields such as TraceId, SpanId, ParentSpanId, ThreadId, and CorrelationId can help follow an execution. These fields describe the documented Azure SRE Agent environment; they should not be assumed to exist identically in other runtimes. See Microsoft’s Azure SRE Agent audit guidance.

3. Confirm the operation in the system that owns the resource

Search the affected service’s audit history for the resource operation and compare it with the agent trace: actor, resource, operation, result, and time. The target system’s own event is the strongest confirmation that the operation was applied. If the action was made through an intermediary, inspect both the intermediary’s record and the resource owner’s record where available.

Azure SRE Agent guidance distinguishes agent action telemetry from Azure Activity Log, which records Azure Resource Manager operations such as creating, updating, or deleting agent-related resources. AWS guidance recommends monitoring agent tool use through CloudTrail and CloudWatch, using metrics and alarms to identify deviations, and aggregating logs centrally to correlate patterns across sessions and users. The appropriate confirmation source depends on the target service. See Azure SRE Agent audit guidance and AWS guidance for generative AI agents.

Rank #3
Yubico - YubiKey 5C NFC - Multi-Factor authentication (MFA) Security Key and passkey, Connect via USB-C or NFC, FIDO Certified - Protect Your Online Accounts
  • POWERFUL SECURITY KEY: The YubiKey 5C NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
  • WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5C NFC secures 100+ of your favorite accounts, including email, password managers, and more
  • FAST & CONVENIENT LOGIN: Plug in your YubiKey 5C NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
  • MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
  • PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts

4. Compare what happened with what was authorized

Compare the requested task with the actual tool call and arguments, its result, the acting identity and permissions, any approval decision, and the target-side operation. Ask whether the tool was available to that agent, whether a human approval was expected, and whether the effective role granted more access than the task required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

AWS recommends least privilege for agent roles and warns that broad permissions combined across tools can create privilege-escalation risk. OpenAI’s Codex safety article describes activity exports that can include approval decisions, tool results, MCP use, and network proxy allow-or-deny events; that exact event set should not be generalized to every agent product. See AWS agent security guidance and OpenAI’s account of running Codex safely.

Which log answers which question?

No single record necessarily shows the full path. Use the source responsible for each evidence question, and keep its limits attached to the finding.

Rank #4
Yubico - Security Key NFC - Basic Compatibility - Multi-Factor Authentication (MFA) Key, Connect via USB-A or NFC, FIDO Certified
  • POWERFUL SECURITY KEY: The Security Key NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
  • WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key NFC secures 100 of your favorite accounts, including email, password managers, and more.
  • FAST & CONVENIENT LOGIN: Plug in your Security Key NFC via USB-A and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
  • TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
  • BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
Evidence source Useful for Important boundary
Agent runtime trace Session, turn, model and tool spans, recorded arguments and results, errors, and available correlation identifiers. OpenAI tracing supports session trace export as OTLP JSON when enabled and authorized: OpenAI tracing. A trace documents runtime activity; it does not independently prove that a target-side change persisted.
Identity and sign-in records Which identity type initiated or performed activity, affected target resources, agent type, blueprint association, and sign-in context. See Microsoft Entra Agent ID logs. Sign-in events can be distributed across log types, and actor labels should not be treated as interchangeable.
Target service audit records Whether the resource-owning service recorded the create, update, delete, or other relevant operation. Azure Resource Manager actions, for example, are recorded in Azure Activity Log; AWS guidance points to CloudTrail and CloudWatch monitoring. See Azure guidance and AWS guidance. Choose the audit source for the actual target service; agent telemetry alone is not a substitute.
Compliance audit and content review Audit operations and, through a separate content-review process where authorized, prompt or response evidence. See Microsoft Purview audit guidance and the Microsoft AI Investigation Playbook. Metadata-first audit events may establish who did what, when, and with which resources without containing prompt or response text.
Security or activity export Approval, tool-result, MCP, and network proxy decision events in the Codex activity-export context described by OpenAI. See OpenAI’s Codex safety article. Do not assume another agent product exports the same event categories.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Correlate records without overstating certainty

Build one timeline containing identity, sign-in, agent trace, approval, network, and target-system events. Preserve each source’s identifiers, including trace, span, session, thread, call, actor, and resource IDs. Correlation IDs are platform-specific; a shared value is useful evidence, not a universal field or guaranteed join.

  • Mark a join as confirmed when records share a reliable identifier or other direct linkage.
  • Mark a join as inferred when it relies on a combination such as actor, resource, and a narrow time window.
  • For an inferred join, state the basis and any alternatives; do not present it as deterministic.
  • Note known timezone, clock, ingestion, export-time, and retention limitations.

OpenAI notes that traces become available after turns finish and that an export includes traces available at export time. Microsoft notes that provider or model details may be absent for some automatic routing cases. Those timing and metadata limits can affect how complete a reconstructed timeline appears. See OpenAI tracing and Microsoft Purview audit logs.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

When the investigation needs prompt or response content

First decide whether content is necessary to answer the incident question. An audit record may establish who acted, when, and on which resource without revealing the prompt or response. Microsoft’s investigation playbook directs content-level review to Purview eDiscovery or DSPM for AI when needed; that review may require additional permissions and legal coordination. OpenAI traces can include generation inputs and outputs in some circumstances, but visibility depends on platform recording and organizational controls. See the Microsoft AI Investigation Playbook and OpenAI tracing documentation.

Contain the path and recover the resource

After you understand the action and its scope, use the organization’s incident process to constrain the relevant identity or tool path, assess impact, and restore the target resource through its approved change process. Least-privilege permissions, centralized logs, and anomaly monitoring can reduce exposure and improve detection, but logging by itself does not prevent unwanted actions. The appropriate rollback or restoration procedure depends on the target service and the specific change; preserve both the action history and restoration evidence in the incident record.

Choose an audit approach that covers the whole chain

When evaluating an organization’s audit coverage, check whether it can attribute identities, show model and tool activity, confirm resource-side operations, expose content when authorized, retain and export records, and correlate with the organization’s cloud, identity, and SIEM systems. OpenAI tracing, Entra identity logs, Azure SRE Agent telemetry, AWS CloudTrail and CloudWatch guidance, and Purview audit and content workflows cover different parts of this chain rather than providing one universal record.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.