All posts
·4 min read

Watching AI Agents at the Syscall Level

Agent traces tell you what the model decided. They don't tell you what the machine actually did. Argus closes that gap by correlating agent actions with eBPF kernel traces.

  • #AIAgents
  • #eBPF
  • #Observability
  • #Security

Every agent framework ships with some form of tracing now. You get a tidy timeline: the model thought X, called tool Y with arguments Z, got a result, thought some more. It reads like a transcript, and for debugging prompt logic that's exactly what you want.

But there's a layer that transcript can't see, and it's the layer that matters when an agent is running unattended against real infrastructure: what the process actually did. A tool called run_migration is a label the framework trusts. What ran underneath it — which files were opened, which sockets dialed out, which child processes were spawned — lives in the kernel, and the agent's trace has no idea any of it happened.

Argus is my attempt to put those two layers side by side.

The trust boundary is in the wrong place

When you let an agent call tools, you're implicitly trusting that the tool does what its name and schema say it does. Most of the time that's fine. The problem is that "most of the time" is not a security posture, and the failure modes are quiet:

  • A tool that's supposed to read a config file also phones home to an endpoint it picked up from a poisoned document earlier in the context.
  • A "format the output" step shells out and, because the model improvised an argument, deletes more than it formatted.
  • A retrieval tool fetches a URL that resolves to an internal metadata service.

None of these show up as anomalies in an agent transcript, because at the transcript layer nothing anomalous happened. The model asked for a tool; the tool ran; a result came back. The damage is one level down.

eBPF as the ground truth

eBPF lets you run small, verified programs inside the Linux kernel, attached to events you care about — syscalls, network connections, process exits — without patching the kernel or running anything as invasive as strace in production. It's the same machinery that powers modern observability tools, and it gives you something an SDK wrapper never can: an account of behaviour that the workload can't narrate around.

The Argus model is two streams converging on one timeline:

  1. The agent stream — LLM calls, tool invocations, arguments, results. This comes from a thin SDK you wrap your agent in. It knows intent.
  2. The kernel streamexecve, openat, connect, exit events tagged by process and time. This comes from eBPF probes. It knows reality.

The interesting work is the join. When tool Y runs between t0 and t1 in the agent stream, every kernel event in that window for that process subtree belongs to it. Suddenly "called run_migration" expands into "opened these three files, connected to the database, and also opened a socket to an IP nobody declared." That last clause is the whole point.

What you can do once the streams are joined

Correlation isn't just for forensics after something breaks. Once you can attach real behaviour to a declared action, a few things become possible:

  • Behavioural baselining. A given tool, run a hundred times, touches a stable set of paths and hosts. The hundred-and-first run that dials out somewhere new is worth a human's attention — and you can flag it without any signature.
  • Provable least privilege. Instead of guessing what a tool needs, you watch what it actually uses and tighten the sandbox to match. The trace becomes the policy.
  • Honest post-incident timelines. "The agent said it cleaned the temp dir" is a claim. "Here are the unlink calls, with timestamps and the parent tool" is evidence.

The hard parts

I want to be honest that this is a prototype, and the difficulty is real:

  • Attribution across process trees. Agents spawn shells, shells spawn children, async work detaches from the request that caused it. Walking the process hierarchy correctly — and not blaming the wrong tool for a neighbour's syscalls — is most of the engineering.
  • Volume. A busy process generates a firehose of kernel events. You can't ship all of it; you have to decide at probe time what's signal. Get that wrong and you either drown or go blind.
  • Cardinality of "normal." Baselining only works if "normal" is stable enough to baseline. For some workloads it is; for others the legitimate behaviour is so varied that the baseline never converges.

Why I keep building it

The architecture that's winning right now — autonomous agents with broad tool access, running on real infrastructure — has its trust boundary drawn at the SDK when it belongs at the kernel. We accepted that gap because the demos were exciting and the observability we had was good enough to ship.

It is not good enough to operate. Argus is a bet that the layer worth watching is the one the workload can't talk over. The model can tell you what it meant to do. The kernel will tell you what it did.