Skip to content

Topic

Operations

Running software in production: Linux servers, deployments, backups and reliability.

Rust

Cargo's defaults ate 483 GB: a Rust build postmortem

A postmortem of build storage and test time across seven Rust workspaces: a 483 GB target folder that stalled a 32-core desktop on disk I/O, a release that ran another checkout's tests, a /tmp full of test leftovers, and the changes that fixed each one, with the numbers.

Read

Operations

Keeping a daemon alive

Insensical's daemon owns every terminal you run, so it mustn't die and it mustn't stall. What threatened it wasn't clever bugs. It was fork getting slower with every thread, a signal mask children inherit, a cgroup that gets cleaned up, and a 1 KiB read on a Mac.

Read

Engineering

Seventeen years on both sides of the line

Most organisations separate the people who build software from the people who run it. I have spent my career doing both. Here is what that taught me about designing systems that survive contact with production.

Read

Operations

The last reading before a crash

A monitor is most useful at the exact moment the machine dies — which is when most of them lose their data. Designing Sysman's collector around one rule, and what it took to keep that rule.

Read

Operations

Release scripts that refuse to half-ship

The worst release is the one that stops halfway — a tag pushed, a package published, the documentation still describing last week. Every project I ship uses the same pattern to make that impossible: check everything, ask once, then do it all.

Read

Engineering

Rewrite history only when you can prove it

Nibble had thirty-six database migrations after its first week. Folding them into one per table was easy to type and hard to get right. The order of foreign keys was the whole job - and a schema diff was the only proof I trusted.

Read

Engineering

Designing software that strangers can upgrade

Anything a released version reads at runtime is a contract with the future. What Nibble taught me about upgrades — a release that can't know its own future address, why merging git history was the wrong model, and checksums that refuse to destroy your edits.

Read

Security

Security that lives in the browser is decoration

Nibble's Control Plane locked itself after fifteen idle minutes — and a page reload walked straight back in. Two humbling fixes, and the sysadmin's rule behind them - enforce where the state lives.

Read

Engineering

SQLite is enough (for most of us)

For most of my career, "which database?" meant "which database server?" Today my default is a file. Here is how to run SQLite in production properly — WAL, backups, honest limits — and when to move on.

Read