Case study · Ask AI · Sprinto 2025–26
Conversation as the control plane
Most B2B AI features are a chatbot bolted onto a dashboard. Ask AI is an action layer: it knows what you're looking at, carries the context, and is reused across the product instead of rebuilt per page.
Results
What changed, labelled by how it was verified.
The real problem
AI in B2B usually fails the same way. Users have to discover a feature, then work out what to say to it. The result is a chat box that demos well and rarely gets opened again.
Sprinto also had a mess underneath: a different drawer on almost every page, and drawers stacking on top of drawers. Any AI layer built on that would inherit the mess.
Three decisions
No blank chat box
The drawer opens with a list of actions the system can actually perform on this record. Structure first, open questions second. People learn what it can do before they type anything.
Trade-off: less of the "ask me anything" magic, in exchange for features people actually use. Open-endedness is a capability, not an interface.
Aware of the record, not the page
One component adapts to whatever object you're looking at (staff, risks, vendors, trainings) instead of a custom drawer per screen. That's why it reached 15+ pages without a redesign.
Context is inherited, not asked for. Open it on the vendor page and it already knows which vendors are in scope. It acts; it doesn't interrogate. Along the way we set a rule of one drawer at a time, which removed a whole class of "where am I?" bugs.
Trade-off: a harder component to design and govern, in exchange for one thing to maintain instead of fifteen.
Test it against the old way, don't replace on faith
We rolled it out behind feature flags, with a fallback to the old drawers, so the new model was tested against the old one. Empty states stayed honest: with no staff imported yet, the drawer still opens and tells you to connect an identity provider first.
Trade-off: a slower rollout and two systems running side by side for a while.
What changed
The same action layer now runs on 15+ pages, and 12+ AI patterns live in the design system with their guardrails attached: where the answer came from, how confident it is, and how to get out. If conversation is the control plane, the product also stops being tied to its own screens. The same actions can be called from the app, from Slack, or from an AI assistant.
What didn't work yet
The quality of the output is still in progress. Some answers aren't good enough yet, and in compliance a confident wrong answer costs more than no answer. So we treat quality as a measured thing: how much output ships with a reason (the standard is at least 90%), how often people override it, and whether the action actually saved a step. Each round, we change what those numbers point at.