· 18 min read
Building Chat Memory That Still Works at Turn 900
Replaying the full chat history works at turn 3. At turn 900 it is why your bill is high and your assistant is wrong.
Read on Substack →
ai engineeringsystem design
Featured writing
Writing on AI engineering
Prompt engineering, system design, and building AI products that hold up past the demo. Cross-posted from Substack.
· 11 min read
Your System Already Has an Overload Strategy. You Just Didn’t Pick It.
A plain-English guide to backpressure, the most important concept nobody explains to junior engineers.
Read on Substack →
distributed systemssystem design
· 6 min read
The Cost of Certainty in Distributed Systems
Most distributed systems don’t eliminate uncertainty — they just hide it.
Read on Substack →
distributed systemssystem design
· 11 min read
How LinkedIn Could Feed You a Feed in Under 200ms (An Essay on Scale)
A quick note before we start: this is not a description of LinkedIn’s actual architecture.
Read on Substack →
system design
· 6 min read
The Math Behind Your Feed’s Order (And What It Says About the Product)
Every social feed you scroll runs a scoring function on every post, in real time, before you ever see it.
Read on Substack →
ai engineeringsystem design
· 11 min read
The 8 Constraints You Need Before You Draw a Single Box
“Design Twitter.”
Read on Substack →
system design
· 8 min read
The Interview Question That Actually Tests Seniority (And It’s Not About the Algorithm)
“You have a 20GB CSV.
Read on Substack →
system design
· 6 min read
Lock, check, or reserve: the concurrency decision nobody teaches
Somewhere right now, a transaction is holding a lock open while it waits on a payment gateway to respond.
Read on Substack →
concurrencysystem designlocking
· 5 min read
The Hidden Cost of Chatty I/O
Every network call has a fixed tax.
Read on Substack →
system design
· 8 min read
Before You Scale: What Chapter 1 of Designing Data-Intensive Applications Actually Teaches You
Everyone cites Designing Data-Intensive Applications. It’s on the reading list of every serious engineering team. Engineers drop it in interviews. CTOs reference it in system design discussions. But most people read it once, nod along, and move on without internalizing what the first chapter is actually saying.
Read on Substack →
system design
· 4 min read
Streaming in LLM Agents Isn’t Just a UX Feature. It’s an Architecture Decision.
Most teams add streaming to their AI products because it feels faster.
Read on Substack →
ai engineeringsystem design
· 7 min read
Building Internal Chatbots That Actually Work
Every company I’ve worked with has the same problem dressed in different clothes.
Read on Substack →
ragai engineeringsystem design