Product case · 12 / 13
Changing the engine without stopping the car
Product note. Organizations, dates, private data and implementation details are omitted.
The problem and the decisions
Every time someone opens an account, another company checks in the background that they are who they say they are. We changed which company. It sounds small and it is terrifying: everyone signing up goes through it, and a bad hour means real people locked out of their own account.
We shipped it as a switch we could flip, not as a new version of the app. On for one state, watch how many people get approved, flip it back in seconds if the number moves the wrong way.
In the first state, 10.8% more people got approved, and 15.4% of the people who had been wrongly turned away got in. Nothing else broke. We never had to flip it back, which is exactly why it had to be there.
What it was built with
What I did
What I learned
The value of an undo button is not how often you press it. It is what you are willing to try because it is there.