I Built Something: 6 Hard-Won Lessons From Agents That Kept Breaking

Here is a number I did not expect to be writing down.

Ninety percent. That is roughly how many of the agents I have built are still broken. Not broken on day one, when you would forgive it. Broken now, months in, after real hours, real iteration, and a fair amount of stubbornness.

And I did not fail quietly in a corner. I was loud about it. I spent two weeks on this very blog telling you to stop playing and start producing, to retire your beliefs, to let go of the dashboard, to climb the ladder from observe to act. Confident stuff. Tidy arguments. Then, to really commit to the bit, I set my own team a 30-day challenge: pick something real, build an agent for it, no more talking about AI, just build. I was not going to be the person who only narrates the revolution from the stands.

So we built. For thirty days, and honestly well beyond them. And at the end of the thirty days, about nine out of ten of the things we built still did not work.

I planned to write the triumphant post. This is not it. So let me tell you about one of the ten, the one I most wanted to work, and what its stubborn refusal to work has taught me.

What I set out to build

Here is the simple dream.

Every weekday evening at 6pm, an agent looks at my calendar for tomorrow. It finds the one-on-ones I have with the people-managers I support. And for each of those meetings, it prepares a deep, strategic briefing that quietly lands in my inbox before I log off.

Not a summary. A briefing. The kind of prep I would do myself if I had two uninterrupted hours before every one-on-one, which I do not.

The idea was that the agent, not me, would read the signals I never have time to stitch together. The recent Slack threads. The email traffic. A “Leader Card,” which is an org-health snapshot I pull from our HR dashboards, the kind of thing that tells you how a leader’s org is actually doing underneath the surface. The agent would connect the dots I usually connect myself, and then stop right there. It would explain, not act. It would hand me how to make tomorrow’s conversation count. Where to push. What to ask. What this manager is probably not saying out loud. Everything I needed to walk in and act, without doing the stitching myself.

I wanted to walk into every one-on-one already knowing the terrain. I wanted the number to find me, in the room where the decision was being made, exactly like I told all of you to want last week.

It was going to be beautiful. After all, so many before me have built this.

The mess, and what it taught me

It was not beautiful.

It broke quietly. When the orchestrator went to prepare a briefing, it leaned on four sub-agents at once, and one of them, the “Leader Card,” kept coming back empty. Cleanly and confidently empty, as if there were simply no one to find. So the orchestrator sat and waited for a piece that never arrived, hanging for hours, never coming home. No error. No timeout I could point at. Just silence on the evenings that mattered most.

The irony is not lost on me. I wrote a whole post about how divergence has to converge somewhere, then built a thing that diverged into four agents and never converged at all. I built the busy-but-misaligned trap and put it on a 6pm schedule. It is quietly good at the easy nights, when there is nothing to prep and doing nothing is the right answer. It is the important nights it cannot handle.

This broken, unfinished little project taught me more in a few weeks than the two tidy posts before it did. Six lessons.

Lesson #1: Build the smallest useful thing first.

What happened: I skipped straight to the impressive version. Four sub-agents, a strategic read, the works. I never built the dumb one-agent version that just reads tomorrow’s calendar and writes three plain sentences of prep.

The lesson: The ambitious version is more fun to imagine. The small version is the one that actually ships. If the simplest cut does not work end to end, nothing you stack on top of it will either.

The fix: Ship the boring version first. Get “read calendar, write three sentences, land in inbox” working reliably before you let it dream of anything more. Earn each layer of ambition.

Lesson #2: Your agent is only as good as its weakest dependency.

What happened: The whole briefing leaned on the Leader Card, and the Leader Card was quietly broken. The dashboards it pulls from had hidden default filters. Because of where I sit, location defaulted to Ireland and job family defaulted to software engineering, so any leader who did not fit those defaults came back empty. No error. Just a clean, confident, empty card.

The lesson: The real work of building with AI is not the prompt. It is the data and the defaults underneath it. I kept trying to fix the words I gave the agent. The problem was never the words. It was that the well it drew from was empty and lying about it.

The fix: Before you trust an agent, go and break its inputs on purpose. Check the defaults you did not set. Prove each dependency returns what you think it returns for the cases you actually care about, not just the demo case.

Lesson #3: A helper is allowed to fail. It is not allowed to vanish. (How the parts talk to each other.)

What happened: My agent fanned out to all four helpers at once and then sat back, patient as a saint, waiting for every one to report. The Leader Card never did. Not with data, not with a failure. It just never came back. And a patient orchestrator waiting on a helper that never returns does not fail. It waits, all night, for a knock at the door that is never coming.

The lesson: Waiting for everyone to come back only works if everyone is guaranteed to come back, even if all they bring is “I could not do it, and here is why.”

The fix: If you are going to have one agent hand work out to others and wait, there is a pattern that works, and it has a rhythm to it. The main agent sends the helpers off, then genuinely lets go and waits rather than sitting there holding its breath. Each time a helper reports back, it checks: is everyone home yet? If not, it waits again. Only when all of them have reported, with an answer or an honest failure, does it pull the pieces together and act. And critically, it gives itself a limit. After a few rounds of waiting, it stops holding out for the stragglers, works with what it has, and says plainly what is missing. Spawn, wait, gather, and know when to stop waiting. Two rules make that rhythm safe: every helper must report, success or failure, no vanishing; and the main agent must never wait forever.

But the thing that actually rescued this agent was simpler and, honestly, humbling. I stopped firing all four helpers at once. I had it run them one at a time instead, in sequence, each finishing before the next began. It felt slower, and I braced for it to drag. It did not. For the first time, the thing ran start to finish and landed a real briefing in my inbox. Four agents sprinting off in four directions is how you lose track of all four. One agent doing four things in order is how you keep hold of the thread. The impressive architecture was the problem. The boring one worked.

Lesson #4: Never hand a human a confident blank. (How the system talks to you.)

What happened: Lesson three was about the agents keeping honest with each other. This one is about the agent keeping honest with me. Early on, when a piece of the briefing came back thin or empty, the agent would paper over the gap and send me a tidy, confident-looking email anyway. It looked finished. It was not. And a briefing that looks complete but quietly isn’t is worse than no briefing at all, because I walk into the one-on-one trusting it.

The lesson: Between two agents, silence means one hangs forever. Between an agent and a person, silence is worse: it means I act on a confident blank without knowing it is blank. A system talking to a human must never dress up “I couldn’t” as “here you go.”

The fix: This is the instruction I am proudest of. The agent sends me what it has, but it is loud about what it does not. It never dresses up a gap as a finished answer. The calendar was empty. The Leader Card came back blank. It timed out on the third helper. It tells me exactly which piece is missing and why, so I can work with it to fill the gap or find another way in. A flagged hole I can act on. A hidden one I cannot. If you have read this blog for a while, you know I have a thing about transparency. Turns out that is not just a value. It is also decent engineering.

Lesson #5: Feed machines what machines can read.

What happened: I kept handing my agents Word documents and Excel files, the way I would hand them to a person, and then wondered why the answers came back muddled. The agent was squinting at formatting it was never meant to parse.

The lesson: We reach for the formats that are comfortable for us. A polished Word doc, a neatly tabbed spreadsheet. But the agent is not reading it the way you do. It does better with plain, structured text than with all our human formatting on top.

The fix: Convert the Word document to JSON before you hand it over. Convert the spreadsheet to CSV. It feels like a downgrade, stripping out all that nice formatting, but machines read the stripped-down version far better than the pretty one. The answers got noticeably sharper the moment I stopped feeding it docx and xlsx and started feeding it JSON and CSV.

Lesson #6: Ask for five passes, not one.

What happened: I used to take the first answer the agent gave me, judge it, and move on. If it was mediocre, I assumed that was the ceiling.

The lesson: These things are probabilistic, not deterministic. The first answer is not the best answer. It is just the first one. Ask it to try again, and again, and it keeps getting better, because each pass is a fresh roll rather than a locked result.

The fix: Ask it to take five passes at the answer, then give you the best. Or run it a few times and compare. It sounds wasteful, and it is a little, but the quality climb from pass one to pass five is real and often the difference between “not quite” and “yes, that.”

Where this actually leaves me

I would love to tell you it all works now.

It does not, not all of it. The Leader Card still returns blanks for the very people I am trying to understand, and I am still, this evening, in the logs, chasing that one down. But it is no longer fully broken either. The night the sequential version finally ran start to finish and dropped a real briefing in my inbox, I sat and stared at it like it was a small miracle. One win out of a long list of failures. That is what progress actually looks like, and nobody puts that part in the demo.

But building it taught me more than writing the two polished posts before it did. It is so much safer to write about the revolution than to build one clumsy corner of it and watch it fail on a Tuesday night. Writing lets you sound finished. Building keeps you honest, because the thing either runs or it doesn’t, and no well-turned paragraph will fix a hidden default filter.

So this is where I am. One agent that finally limped to the finish line, a folder full of logs, a challenge that is still mostly unfinished, and a real, first-hand understanding of just how hard producing actually is. I spent a month telling everyone to produce, and then I tried, and it humbled me. What I am left with is genuine appreciation, and if I am honest, a little envy, for the people quietly shipping polished agents that just work. That is a craft. It is harder than it looks, and it is a skill I am determined to keep polishing.