Why a design note should record the measurement, the decision and the trade-off, so nobody reruns the experiment, and a template you can copy.
The most expensive experiment on a small team is the one somebody already ran. They measured it, decided and moved on. Later someone asks why we do it this way, nobody can say for sure, and the cheapest way to find out is to run the test again.
I write design notes to stop that. Each one records three things: the measurement, the decision and the trade-off. The aim is that the next person reads the result instead of reproducing it.
Decisions get written down. Their reasons don’t.
When a team writes a decision down at all, it usually writes the conclusion. “We use X.” That tells you what to do today. It doesn’t tell you whether it’s still true.
A test rarely gets rerun because nobody decided. It gets rerun because the measurement behind the decision lived in one person’s head, a chat thread or a terminal that has since closed. A conclusion without its number can only be trusted or tested again, and careful engineers test it again.
A small team feels this more. Five people can’t keep every decision in every head, and the person who ran the test is usually busy with the next one.
Record what lets someone check you
A note works when someone who wasn’t there can check the decision without redoing the work. That takes three parts, and each one has its own job.
The measurement lets them check it. What you tested, on what, and what came out. Real numbers, with the setup. A benchmark on a laptop is not a benchmark on the machines you run in production, so say which it was. (Benchmark on the hardware you pay for is why that matters.)
The decision tells them what to do. One or two sentences. What we do now.
The trade-off tells them when to stop doing it. What you gave up, and what would make you revisit it. This is the part people skip, and the part the next person needs most. If the data changes, or the thing you gave up starts to matter, they know the decision is open again and which test to rerun. If a note says a choice cost nothing, either it wasn’t a real decision or the note isn’t finished.
A note about something you decided not to build counts too. “We tried it, here’s the number, it wasn’t worth it” may be the most useful note a team has, because that’s the experiment people keep wanting to run again. AI experiments we decided not to ship is that kind of result.
Smaller than a design doc
A design note is not a design doc in the big-company sense. There’s no ten-page proposal and no review meeting. It’s closer to an architecture decision record (ADR), a format Michael Nygard described in 2011 for recording one decision with its context and consequences.
The difference is the starting point. An ADR starts from context. A design note starts from the measurement, because on a small team a test settles more decisions than a meeting does.
Size matters for a plain reason. A note that takes a day to write gets written a few times a year. A page you fill in right after the experiment, while the numbers are still on screen, gets written every time.
Notes are context for agents too
AI coding agents start each session knowing nothing about your codebase or its history. A written decision is context an agent can read. An agent asked to speed something up, with a note in front of it that says “we tried this, it cost accuracy, here’s the number”, is less likely to propose the thing you already rejected.
It’s the same idea as our agent kit: write it down once, well enough that a person or an agent can act on it.
How to write one
Copy this, fill it in the same day, and keep it to about a page.
# <The decision, in one line>
Date: YYYY-MM-DD
Status: decided | superseded by <link to the newer note>
## Question
What were we deciding, and why now?
## Measurement
- Setup: data, hardware, versions
- What we compared
- Results: the numbers, with units
## Decision
What we do now, in one or two sentences.
## Trade-off
- What we gave up
- What would make us revisit this- One decision per note. Title it with the decision, not the topic: “Retry failed webhooks three times”, not “Webhooks”.
- Write it the same day. Memory rounds numbers off. A note written a week later is a guess.
- Never leave the trade-off empty. If you can’t name what you gave up, keep thinking before you call it decided.
- Don’t rewrite old decisions. When a decision changes, write a new note and mark the old one superseded. The history of why is part of the value.
- Keep the raw results close. Link the script, the spreadsheet or the log the numbers came from, so anyone can rerun exactly that test if they need to.
- Put notes where people already look. In the repo, they sit next to the code they explain, and an agent working there can find them.
The lesson
Write down the number, not just the conclusion. The conclusion tells the next person what to do. The number tells them when to stop doing it.
Quick answers
- What is a design note?
- A short document that records one engineering decision: what was measured, what was decided and what was given up to get it. Its job is to save the next person from rerunning the same experiment.
- Why do teams keep rerunning the same experiments?
- Usually because the result was written down without its numbers, or not at all. A conclusion on its own can only be trusted or tested again, so careful people test it again.
- What should a design note include?
- The question being decided, the measurement with its setup and numbers, the decision in one or two sentences, and the trade-off: what you gave up and what would make you revisit it.
- What is an architecture decision record (ADR)?
- A short document that records one architecture decision with its context, status and consequences. Michael Nygard described the format in 2011.
- How long should a design note be?
- About a page, short enough to write on the day of the experiment while the numbers are still in front of you.