Lampson's Hints Distill System Design Into Timeless Principles

April 29, 2012

In 1983, Butler Lampson published "Hints for Computer System Design" a paper that reads as well today as when it was written. Its longevity comes from addressing the invariant truths of engineering: that design is about tradeoffs, interfaces outlive implementations, and simplicity is the hardest thing to achieve.

"Writing is nature's way of showing you how fuzzy your thinking is." Guindon, via Lampson

The core insight is that engineering is compromise. You may want your system to be simple, dependable, scalable, efficient, gracefully degradable, fault-tolerant, and highly responsive. Trying to achieve all at once almost certainly results in failure. Choose your battles. The absence of clearly prioritized goals leads to the temptation of optimizing everything, which optimizes nothing.

Lampson's most durable hints: Keep interfaces simple an interface should capture the minimum essentials of an abstraction. When an interface undertakes too much, the result is large, slow, and complicated. Get it right first, then make it fast neither abstraction nor simplicity substitutes for correctness. Use a good idea again instead of generalizing it a specialized implementation is often more effective than a generic one. When in doubt, use brute force a straightforward solution requiring lots of compute beats a complex, poorly characterized one that may work if certain assumptions hold. Safety first in allocating resources, avoid disaster rather than attain the optimum.

On fault tolerance: end-to-end error recovery is absolutely necessary for a reliable system, and any other error detection or recovery is strictly for performance. Make actions atomic or restartable. Make APIs idempotent. Log updates to record truth about state.

The meta-lesson is about intellectual humility in the face of complexity. Predicting the future is hard, so plan for success and change. Successful systems last a long time and eventually must scale. Failures are inevitable embrace them and make them cheap.

Takeaway: The best system design advice fits on a few pages because the hard problems are not technical but judgmental knowing which tradeoff to make, when simplicity matters more than power, and when brute force is the elegant solution.