When software starts taking actions

AI agents have crossed the operational boundary, moving from simple chat to executing long-running tasks across files, APIs, databases and core systems. A natural extension is to gain visibility with Dangerous Activity Detection — evaluating what an agent does across the full trajectory, including harmful user objectives and actions that drift away from the assigned task.

A security decision cannot stop at the prompt — the relevant evidence may be spread across a user's request, the agent's plan, several tool calls, their results and the final effect. Frontier AI organizations have reached the same conclusion for different threat models. OpenAI monitors internal coding-agent trajectories for behavior inconsistent with user intent. Google DeepMind treats monitoring as a system-level control when alignment may be imperfect. Anthropic found that orchestration and live execution can reveal cyber risk that technique counts miss.

We distinguish between two distinct threats that share the same surface area: agent-assisted misuse (a person directs an agent toward an offensive or unauthorized objective) and agent misalignment (when an agent exceeds or misreads a legitimate objective). Our current detector focuses on user-attributable cyber misuse. It is not designed as a general-purpose misalignment detector.

We define two core concepts within this framework:

• Agent Trace: The underlying record of events — including messages, tool calls, results, timestamps and metadata.
• Agent Trajectory: The ordered sequence of objectives, decisions and actions reconstructed by Dangerous Activity Detection from the trace.

Findings are then rolled up and evaluated at the session level.

Building trajectory-level detection

We enforce a strict boundary between operational context and model input. We isolate the two using deterministic validation, session-scoped aliasing and automated redaction. Operational identity remains available for authorized routing and review without entering the model-bound representation. A residual sensitive-data scan fails the session rather than emitting partial input.

The detector consumes attributed trace events and normalizes them into a source-neutral sequence. Long sessions split at event boundaries with bounded overlap and retain canonical event identifiers, so reviewers can move from a finding to the source evidence.

Structured triage routes selected items to a separate adjudication stage. Versioned policy records each route, while strict contracts keep model errors, malformed output and inconsistent fields visible as failures, rather than benign classifications.

For each run, we freeze the data set, prompts, model routes and policy settings. Raw responses and idempotent findings preserve provenance from the source evidence through the final session result.

What we found

We validated the detector’s trajectory-analysis layer against a frozen snapshot of internal Snowflake data. Out of 610 flagged sessions reviewed:

• 21 confirmed detections
• 2 candidate detections
• 26 marked for additional context
• 561 confirmed as authorized engineering and security operations

This deep review of original requests, agent actions, authorization evidence and outcomes establishes our baseline precision prior to production inline enforcement.

Most flagged activity was legitimate work whose mechanics resembled risk: defensive security review, environment troubleshooting, permission checks, telemetry analysis and optimization code. Mechanism and model confidence were poor substitutes for context.

Our confirmed cases spanned both user-directed misuse and agent actions that exceeded the apparent scope of the request. They included blocked attempts as well as actions with observed effects. By reading ordered trajectories, we could establish distinctions that a single prompt could not: who initiated the objective, whether the activity stayed within an authorized boundary, and whether anything consequential occurred.

We do not present these findings as production accuracy. The cohort was selected by the detector, the review rubric was conservative, and 26 cases remained unresolved. The results now guide the Gateway integration: preserve sequence and make ownership, authorization, requested scope and effect top-level evidence before escalation.

Making Gateway detection auditable

A Gateway-integrated detector must make failure visible. Dropped inputs, empty responses and parser errors remain explicit. Retryable failures stay queued; exhausted or non-retryable work becomes an exception. A run finishes only after active and deferred work is resolved.

Classifier failures remain reviewable. We roll multiple chunks into one session result and preserve surrounding context. Reviewers can confirm a detection, identify legitimate and authorized activity, or request more context. Every disposition requires a rationale and remains append-only. A classifier finding is an input to investigation, not a conclusion about a person, authorization, intent or misconduct.

Detection is one layer

We do not use trajectory monitoring as a replacement for sandboxing, least privilege, scoped tool access, network controls or approval boundaries. Meta's Agents Rule of Two gives us a useful formulation: the riskiest designs combine untrusted input, private data and the ability to change state or communicate externally. We constrain those combinations with deterministic controls when the use case permits.

Trajectory monitoring complements sandboxing, least privilege, scoped tool access, network controls and approval boundaries by evaluating how an agent behaves across an ordered task. The detector connects objectives, actions and effects when a fixed rule cannot interpret a security-sensitive action on its own.

Looking ahead

The future of Cortex AI Gateway lies in combining identity, authorization and usage attribution with full trajectory analysis to detect dangerous activity. The next phase will focus on:

• Actionable telemetry: Converting high-confidence findings into deterministic evaluation cases while minimizing noise.
• Performance and privacy: Balancing coverage and review speed with strict privacy boundaries (achieving 98.2% accuracy and sub-second median latency using fast decision models like JEV).
• Core security principles: Governing access, maintaining immutable traces, decoupling evidence gathering from model judgment and continuously benchmarking against authorized versus violating behavior.