Datadog Illuminated: Evaluating an Agent That Creates and Tunes Monitors, with Eric Karschner

kagent is a framework to help you make AI agents and run them in Kubernetes declaratively, with YAML.

Lin Sun, co-creator of kagent, joined to walk through how it works. Here’s the lightboard we made together.

Read the notes: gist.github.com/wiggitywh…

A hand-drawn lightboard diagram on a black background, filled with colorful marker text and boxes mapping out how kagent works. Sections include "Before kagent," "kagent config" with a Kubernetes CRD circle, "kagent controller watches," "Agent Sandbox," "Agent Runtimes," "Why run your agent on Kubernetes," "AUTH," and "How do people use kagent," connected by arrows and underlines in pink, yellow, and green.

Datadog Illuminated: Evaluating an Agent That Creates and Tunes Monitors, with Eric Karschner

Will AI help or hinder a developer’s experience with an internal platform? At KubeCon EU, the audience found out. My bestie Viktor Farcic (Upbound) and I ran an interactive session where attendees voted at key moments on which tools our AI agent would use. I built the demo to work with any combination they chose, on the fly.

The agent investigated a broken app, used semantic search to navigate cluster resources, and deployed on a developer’s behalf, with no cluster access required. Expect surprises, stumbles, and a real conversation about what AI is (and isn’t) ready for in platform engineering.

Watch the recording here: youtu.be/k7ct4sW-9…

Title slide on an orange background. Large bold white text reads "Choose Your Own Adventure: AI Meets Internal Developer Platform." Below in white italic: "Whitney Lee, Datadog & Viktor Farcic, Upbound." The KubeCon and CloudNativeCon Europe 2026 logos appear in the upper left corner. A stylized windmill and Dutch tulips decorate the right side and bottom corners.

Software Defined Interviews: DevOpsDays, Technical Storytelling, and Loading the Dishwasher, with Jason Yee

Datadog Illuminated: Building an Eval Platform for an Investigation Agent, with Benjamin Barton

In this Datadog Illuminated episode, Scott Yak & I discuss how evals for the Datadog MCP server run against live data that goes stale FAST

How did Scott & team solve this?

See the lightboard below & read the board notes here: gist.github.com/wiggitywh…

Handwritten green, yellow, and white notes on a black lightboard titled "Evaluating the Datadog MCP Server." Left column shows "Why eval MCP server?" with a diagram of a query flowing from a person through an Agent (LLM), Datadog MCP, and Datadog Backend and back. Below it, "How eval MCP server?" defines eval = Agent harness + MCP server + Q&A pair, with Actual ≈ expected response (if passing). Middle column shows "Automate eval generation," a flow starting with a seed query fanning out into "FUZZING" to make many question variants, with a note that one seed produces many Q/A pairs and evals get re-run to update answers. Right side lists "Benefits of Great Evals" in four numbered sections: Speed (fast to generate, fast to run, see impact quickly), Visibility Into Progress (high quantity of evals, tagging, traces), Mutual Benefit (Datadog AI agents like Bits AI SRE and Bits AI Assistant use the Datadog MCP, so a better MCP server means better agents), and Devs Like Writing Evals (fast feedback, showing impact, preventing regression).

AI Engineer World’s Fair: Build a Platform, Unleash an Agent on it…. and Watch it Burn!

At Cloud Native AI Day (KubeCon EU), Thomas Vitale (Systematic) and I co-presented a live demo where audience votes drove real-time canary rollouts for GenAI apps using OpenTelemetry and Flagger.

Watch the recording here: youtu.be/CRcYbl-g3…

Title slide with a light blue gradient background. Large dark blue bold text reads: Rollout on Reception: Progressive Delivery for GenAI Apps Using Real-Time User Feedback. Below in dark blue italic: Thomas Vitale, Systemic and Whitney Lee, Data Dog. In the upper right corner: the Cloud Native AI + Kubeflow Day Europe logo, a hexagonal badge with a brain icon and EUROPE in a blue box.

AI Engineer World’s Fair: Build a Platform, Unleash an Agent on it…. and Watch it Burn!

Software Defined Interviews: DevOpsDays, Technical Storytelling, and Loading the Dishwasher, with Jason Yee

Software Defined Interviews: Running D&D with AI, Determinism First, and Workflow ROI, with Michael Rishi Forrester

Datadog Illuminated: Building an Eval Platform for an Investigation Agent, with Benjamin Barton

SREday Austin: Livin' In the Future: Your Platform’s Next Interface Is an AI Agent

Datadog Illuminated: How to Use AI to Find Security Flaws in Static Code, with Bahar Shah

This is the finished lightboard from my 🌩️Thunder conversation with Volkan Özçelik about SPIFFE and SPIRE.

Bottom line: SPIFFE is a secure and automated way to manage identity, no secrets managers needed.

How does it work? gist.github.com/wiggitywh…

A black lightboard covered in colorful handwritten notes organized into four columns. The leftmost column, headed "Before SPIFFE...", lists problems with service tokens and mTLS certificates and defines SPIFFE and SPIRE. The second column, headed "WHO + WHAT = POLICY," includes a small diagram of pods stacked above a host layer and a machine layer, illustrating where identity is best assigned. The third column, headed "SPIFFE is designed to solve these problems," lists SPIFFE IDs, SVIDs, and how the Workload API attests and issues identity to workloads. The rightmost column covers what happens once attestation completes, a boxed list of "Benefits of SPIFFE," and boxed definitions of SPIRE Server and SPIRE Agent.

Software Defined Interviews: The Chief Therapy Officer, Job Titles as Culture Change, and AI as Spreadsheets, with Bryan Ross

“We go through our Kubernetes clusters and we’re killing nodes every 2 minutes in production constantly.”

At Google Cloud Next, I hosted a fireside chat with David Faircloth (VP of Architecture and Engineering at Wendy’s) about how Wendy’s scaled to 99.95% availability in five years.

Wendy’s built their way there by designing systems to fail gracefully. Observability is what made that strategy visible, and chaos engineering is what proved it out in production.

Watch the recording here: youtu.be/UQmFnkZHF…

Talk title "How Wendy's got to 99.95% availability (and how you can too)" in large black text on the left. Top right: photo of Whitney Lee, a woman with reddish-brown hair smiling in a blue top against a colorful lit background. Bottom right: black-and-white headshot of David Faircloth, a man with short hair smiling in a suit jacket. Bottom left: Google Cloud Next 26 logo. White background with decorative geometric shapes and a blue wave design.

“I just Googled Kubernetes and I know more than you do.”

An investor actually said that to Emily Long’s all-woman founding team during a pitch!

In this episode of Software Defined Interviews, Emily Long, CEO and co-founder of Edera, talks about building a hardened container runtime that secures infrastructure foundations instead of chasing detect-and-respond alerts, and what it’s like raising a deep-tech Series A when investors have opinions like that one.

Give it a listen! www.softwaredefinedinterviews.com/122

Software Defined Interviews podcast cover art. Whitney Lee (dark hair with bangs, laughing and clapping, black top) is in the upper left, and Coté (bearded man in a blue plaid shirt, smiling) is in the upper right, both flanking the show's cloud logo. The show name "Software Defined Interviews" appears in large black and white block letters with a red script "Interviews" overlay. Below, Emily Long (long brown hair, brown leather jacket, smiling) appears with an arrow and the text "with guest Emily Long."

At Observability Day, KubeCon EU, I presented Spinybacked Orbweaver, an AI agent that automatically instruments your JavaScript code with OpenTelemetry using OTel Weaver as the standards contract.

Watch the recording here: youtu.be/JgzqzAYL0…

Title slide with a dark crimson background. Large white bold text reads "When the Codebase Starts Instrumenting Itself." Below in white italic text: "Whitney Lee, Datadog." In the upper right corner, a rectangular badge reads "Observability Day EUROPE."