The problem
After a phone call, the customer and the pro each get a short AI recap in their messages. Both recaps have to tell the same story. If the pro’s says Thursday and the customer’s says Friday, someone shows up on the wrong day.
People also trust what’s in the thread. A price in a message preview reads like a promise, and a personal detail repeated by AI can feel like a breach.
The decision: refine after launch, or pause and fix a privacy risk first
We were still testing, and the plan was to keep refining the prompt after launch, which is common with AI features. The day before launch, I called for one more round of testing. My applied scientist and I built a way to check whether the AI would repeat sensitive personal details from a call. In a few test samples, it did.
- The option I passed on: launch on time and keep refining the prompt. Each new prompt rule cut leaks a little but made summaries worse elsewhere, and prompt rules alone would never catch every case.
- What I chose: pause the launch and fix it at the source. I brought Legal clear examples and worked with engineering to remove sensitive details before the AI ever sees them.
- The cost: the launch moved a few days. We’d planned for delays.
How we kept quality high
- Evals for every version. I scored each prompt version against a set of test calls, with several independent AI judges per call, and kept a changelog for each version.
- Errors sorted by harm. A clunky sentence can wait. A wrong day, a made-up price, or a leaked detail blocks the launch.
- People in charge. When AI refined the prompt on its own, it drifted from what the product needed, so every change went through human review.
- Plain rules. Same story for both sides. A takeaway, not an overshare. No names, gender, or contact details.
- Knowing when not to ship. The newest prompt leaked less, but overall quality dropped in evals. We shipped the version before it and built better safety rails around it.
What happened
The final round of testing came back clean, and the feature launched to all users. Now, 90%+ of the time, the customer’s and pro’s recaps tell the same story. AI doesn’t always get it right, so we built in a feedback system and keep testing.
What I delivered
- Content designs for the message card, call details, and feedback flow
- The summary prompt across many versions, with a changelog for each
- Evals for every version, rerun the final week before launch
- A privacy risk review for Legal
- A recurring cross-team prompt crit and a handoff doc