Adventures in DevOps

Share Episode

Welcome back to another hopefully, relief from architectural existential dread. This week, we've pulled in Seth Eliot from Arpio, (Ar-Pi-O, RPO, get it?), to dive headfirst into the beautiful, deeply expensive illusion that migrating your legacy infrastructure to a major hyperscaler magically grants it instant immortality. It doesn't. We break down the shared responsibility model for resilience, which was conveniently cribbed straight from the security model, and analyze how the foundational promise of automated fault isolation boundaries routinely crumbles.

From cloud providers sticking multiple "independent" availability zones inside the exact same physical building, to multi-AZ cascading anomalies, to regional power grid failures, it's clear your provider's abstractions aren't nearly as resilient as their marketing slides suggest.

Discussed within is the "Thundering Herd" phenomenon, that can't be ignored even when the failover clusters are designed correctly. From cross-organization KMS re-encryption loops to the horror of fragmented application logs across CloudFront edge regions, at the end of the day, true resilience isn't achieved by forcing your engineering team to implement features, it's about architecting your baseline, confidentiality for the inevitability of production burning to the ground.

💡 Notable Links:
✨ Episode: Eat your security vegetables
✨ Episode: Matt vibecodes
✨ Episode: on DNS and isolation
🎯 Picks:
Warren - Book: Moldable software development
Seth - Lockpick set

Share Episode

This week's adventure tackles the absolute absurdity of modern enterprise infrastructure, where a single company can easily find itself running multiple different CI/CD platforms due to unchecked mergers and acquisitions. We've brought in Chris Farris, AWS Security Hero and consults with companies via Securosis. And dig deep to find the security cracks and philosophize about the real world impacts of tech debt in the AI age.

Management rarely prioritizes standardization, leaving security teams to defend a chaotic swamp of mixed cloud providers, GitHub repositories, and nostalgic on-prem Bitbucket instances. We define this accumulated technical debt not as some abstract concept, but as literal potholes on the infrastructure Autobahn—annoying speed bumps that permanently damage velocity and set organizations up for an inevitable disaster. We contrast this with the evolution from old-school sysadmins cutting their fingers on rack screws to modern engineers spinning up entire architectures with a few lines of code, noting that the ease of deployment has far outpaced our willingness to clean up our own mess.

The crisis is only accelerating now that the cost of writing code (but not having to maintain it) is rapidly approaching zero. While letting an AI agent autonomously build a website or manipulate an AWS sandbox over a single Saturday afternoon sounds magical, it creates a terrifying volume of unreviewed, context-devoid software. Compounding this systemic frailty, massive cloud provider layoffs mean the crucial institutional memory and human operational experience required to survive are walking right out the door. We expose the fundamental flaw of modern agentic tooling: they completely lack fine-grained access control, operating on a dangerous all-or-nothing identity model. Until autonomous agents are engineered with actual conscience, consequence, and common sense, security teams will continue fighting a losing battle against a digital supply chain.

💡 Notable Links:
Chris' Article on AI Tech Debt
Breaking Open Source: Malus - Article
Vercel Security Incident
✨ Episode:
🎯 Picks:
Warren - Rick & Morty S02 + S03
Chris - Risky Business: The latest actually good cybersecurity news

Share Episode

We grabbed Donald Nguyen, co-founder and CTO at Corvic, to discuss the absurd complexities of enterprise data and multimodal inference. We explore how organizations habitually hoard mountains of useless, "dead" data just out of the sheer fantascy that someone might ask for it later. We highlight the fundamental disconnect where data collectors using tools like Airbyte and Kafka speak a completely different language than the business consumers analyzing it in Excel.

True scale isn't just about managing petabytes; it's the absolute nightmare of extracting subjective business meaning from flat PDFs and invoices. In the deep-end of vector embeddings, we're challenging translating data into a different semantic universe requires imposing a heavy business bias. Auditors and artists will view the exact same invoice completely differently, meaning your embedding model selection is incredibly subjective to the business context.

The industry's desperate search for actual AI success stories beyond basic workflow automation is still ongoing as we laugh—and cry—at the reality that companies are likely budgeting 50% of an engineer's salary for LLM token usage, effectively enabling product managers to burn cash on infinite loops to generate prototype code. Reasonable or unreasonable?

And lastly, we tackle the existential dread of securing autonomous AI agents. Because fine-grained access control for agent actions is basically an unsolved fantasy, we must treat their execution environments as entirely untrusted, relying on rigid sandboxes like AWS Firecracker VMs. Prompt injection attacks are an inevitable flaw of the transformer architecture, and the industry's best defense mechanism seems to be wrapping models inside of other models to validate the outputs. It is quite literally turtles all the way down, and the winner of enterprise security is simply the organization that manages to put one more turtle ahead of the attackers.

💡 Notable Links:
Kuuk Thaayorre Aboriginal Tribe - Cardinal Directions
✨ Episode: Generating automatic integrations at scale
🎯 Picks:
Warren - Dr. NEMO: Clockwise circle pit
Donald - Book: InvestiGators

Share Episode

Down to business with GitHub's Cassidy Williams, Senior Director of Developer Advocacy at GitHub, where we try to untangle the existential dread of modern software development. It includes the sheer absurdity of managing a platform that officially crossed the one billion commit mark in 2025. Currently absorbing a completely unreasonable 275 million commits per week, GitHub's technical debt is naturally showing its age under the weight of AI agents aggressively creating pull requests. And with company's own copilot advocating for more, we explore the daily reality of being the internet's punching bag during an outage, and how the "Tiny Wins" buy back developer affection by still shipping the critical features.

Which of course is a small signal in the sea of the industry's collective identity crisis: vibe coding and the valley of AI-generated garbage. Discussed is one suggested solution of strongly typed languages which are skyrocketing in popularity because we desperately need rigid guardrails to babysit the hallucinated code our non-human agents are frantically pushing to production. Things have gotten so dire that we commiserate on missing the good old days of Stack Overflow, where instead of a chatbot agreeably telling you your terrible idea is great, a grumpy human engineer would just ruthlessly roast your architecture honestly.

💡 Notable Links:
Cassidy's post on Typed Language
Fermat's Last Theorem
Cassidy's newsletter
Book: 4-Hour Work Week
✨ Episode: Typed Languages
✨ Episode: Vibecoding
✨ Episode: Productivity Isn't Real
🎯 Picks:
Warren - Book: The Light Eaters
Cassidy - Obsidian Offline Wiki

Share Episode

Founder of Bespinian and long-time cloud solutions architect, Lena Fuhrimann, sits down with us to clarify the widespread confusion around serverless architecture. We discuss how serverless is often incorrectly equated solely with Function as a Service (FaaS), when it actually represents a broader spectrum on the abstraction ladder—including managed AI inference, container platforms, and databases.

Lena shares her early career traps of building a fragmented landscape of sixty "nano-services" and explains why starting with a well-architected monolith and progressively breaking out microservices based on distinct resource or lifecycle requirements is a much saner approach. Then we shift to drivers behind cloud migrations, emphasizing that the primary financial benefit of serverless isn't necessarily shrinking the monthly cloud provider bill, but rather optimizing your most expensive resource: engineering time. By offloading mundane infrastructure patching to the cloud provider, teams can focus entirely on delivering tangible business value to customers. But cost is still there too.

We also explore the psychological challenges of adopting new paradigms, sharing a fascinating story of bridging the gap for a VM-loving engineer by introducing immutable infrastructure concepts through Packer and Ansible before fully transitioning them to containers. And of course we tackle the dreaded topic of "cold starts" and why complex workarounds—like building custom Lambda warmers to periodically call APIs—often defeat the core benefits of reduced total cost of ownership.

💡 Notable Links:
Bespinian
Book: Drive — Motivation 3.0
✨ Episode: Typed Languages, Haskell, and building monoliths
🎯 Picks:
Warren - Better thank coffee: Himmelstau tea
Lena - Home Assistant open source project and Awtrix Clocks