Designing a validation strategy for AI in clinical environments.
Some project details have been generalized due to NDA.
Artificial intelligence has the potential to reduce the administrative burden placed on healthcare professionals, allowing them to focus more on patient care. But introducing AI into clinical workflows also introduces uncertainty.
Unlike traditional software, AI-powered interactions are shaped by natural language, professional terminology, environmental conditions, and human behavior — variables that call for a different approach to usability validation. As the Senior Usability Designer on this project, I led the usability validation strategy for the assistant before release, designing an evaluation approach that reflected the complexity of real clinical environments.
Traditional usability evaluations are built to answer whether users can complete their tasks, whether they understand the interface, and where they hit friction. Those questions stayed important — but they were no longer sufficient.
When AI becomes part of the interaction, variability becomes part of the product. The challenge wasn't simply validating whether the product worked — it was designing an evaluation capable of exposing it to realistic conditions before release.
Scheduling support, session observation, and document reviews were carried out collaboratively with other members of the multidisciplinary team.
Formative evaluation rounds, using the production system
Healthcare professionals across multiple clinical specialties
SUS score, indicating strong overall acceptance
Task success across AI-supported activities
How do you validate a product when variability is the point?
One of the biggest shifts in this project was recognizing that traditional usability practices alone wouldn't generate the evidence we needed. Instead of minimizing variability, the evaluation intentionally embraced it — natural language, clinical terminology, professional experience, voice interaction, and realistic workflows became essential parts of the evaluation rather than sources of inconsistency.
Rather than asking users to follow ideal interaction paths, the objective was to observe how the AI behaved when exposed to the diversity that exists in everyday clinical practice.
The validation strategy was conducted across three formative evaluation rounds, using the production system connected to a testing environment. Participants represented multiple clinical specialties, including physicians, nurses, pharmacists, and biomedical professionals. All evaluations were conducted remotely using realistic clinical scenarios and AI-supported workflows.
Each session combined task observation, qualitative interviews, behavioral analysis, and standardized usability metrics — to understand not only task completion, but also user confidence, interaction patterns, and opportunities for product refinement.
Where did automation need to stop and human judgment begin?
The value of the evaluation extended beyond identifying usability issues. Each round generated evidence that informed multidisciplinary discussions throughout product development — supporting collaboration across Design, Engineering, Quality, Regulatory, and Human Factors Engineering, and contributing to product refinement before release.
Rather than treating usability findings as isolated observations, the process focused on transforming user behavior into evidence capable of supporting product decisions, and it fed directly into the Human Factors documentation required throughout development.
Validating AI products requires a different mindset from validating traditional interfaces. The objective isn't simply confirming that users can complete predefined tasks — the evaluation has to intentionally introduce the variability that exists in real-world environments. Language, professional experience, context, and unexpected interaction patterns often generate the most valuable evidence. Designing for AI means designing evaluations that embrace uncertainty rather than attempting to eliminate it.
This project fundamentally changed the way I think about usability evaluation. I no longer see validation as the final step before release, or as a mechanism for measuring usability alone. When strategically designed, usability evaluation becomes a decision-making tool — it helps multidisciplinary teams understand uncertainty, prioritize improvements, strengthen Human Factors activities, and increase confidence before products reach healthcare professionals.
Especially in regulated environments, successful validation isn't about confirming expected behavior. It's about creating the right conditions to learn how people, technology, and context interact in the real world.
