Scenario Practice
Nova generates realistic ResolveIQ customer situations without revealing the solution first. The learner responds as the support specialist.
Case study
Nova — AI Learning & Performance Coach
AI-Powered Performance Support • Scenario Practice • Adaptive Coaching
Nova is an AI learning and performance-support tool built for newly hired customer-support specialists. It provides workplace scenarios, coaching, procedure help, and practice. Nova is also used as an optional support tool inside my NovaDesk Storyline course.
- Instructional Design
- AI Performance Support
- Scenario-Based Learning
- Zapier Chatbots
- Knowledge Architecture
- Responsible AI
Context
The Need
New employees often leave onboarding with basic knowledge but still need support when applying policies and procedures during real customer interactions. Traditional courses cannot anticipate every situation, and employees should not be expected to memorize every rule, escalation path, or troubleshooting step.
I wanted to create a tool employees could use when they need help and to give them a place to practice difficult workplace decisions.
Approach
The Solution
I designed Nova as an AI Learning & Performance Coach for ResolveIQ Technologies, a fictional SaaS organization. Nova uses approved ResolveIQ knowledge sources to provide just-in-time procedural guidance, generate realistic customer-service scenarios, coach learners on their responses, and reinforce knowledge through retrieval practice.
Unlike a general-purpose chatbot, Nova is designed around workplace performance. Its responses are based on an approved knowledge base and a concise performance-support job aid.
Project connection
Nova in a Storyline Course
Nova is also used in my NovaDesk: Account Security Escalation Storyline course.
Inside the course, Nova appears as a support tool within the NovaDesk work environment. After the learner completes the performance assessment, an Ask Nova button opens the live chatbot for additional help or practice.
This shows how the same support tool can work on its own and as part of a larger training experience.
Live prototype
Try Nova Live
Step into the role of a newly hired ResolveIQ customer-support specialist. Ask Nova for help, practice a difficult customer interaction, or test your knowledge.
Try one of these:
- Practice a realistic customer scenario.
- Help me with a ResolveIQ procedure.
- Quiz me on ResolveIQ policies.
- Coach me on a customer response.
Architecture
How Nova Works
Step 01Learner Question or Scenario
Step 02Nova AI Coach
Step 03Approved ResolveIQ Knowledge Sources
Step 04Procedure-Based Support & Coaching
Step 05Practice, Feedback, and Improved Performance
Nova uses two intentionally designed knowledge sources: a detailed customer-support knowledge base and a concise quick-reference job aid. These sources contain approved procedures for billing, account security, missing data, troubleshooting, escalation, and customer-service standards.
Design
Learning Experience
Adaptive Coaching
Nova evaluates the learner's response against approved ResolveIQ procedures, identifies critical errors, explains why they matter, and provides another attempt when appropriate.
Just-in-Time Support
Employees can ask procedural questions while working and receive concise guidance based on approved organizational resources.
Retrieval Practice
Nova can quiz learners one question at a time and provide immediate corrective feedback.
LEARNING SCIENCE
Evidence-Informed Design
Nova uses ideas from instructional design, workplace learning, and the learning sciences: realistic practice, corrective feedback, reflection, and support for using new skills at work. These ideas also informed graduate work in my University of Memphis M.S. in Instructional Design and Technology program.
Nova has not been experimentally validated. This prototype shows how those ideas can be used to create practice, feedback, reflection, and support when employees need it.
Authentic & Experiential Practice
Learners build capability by applying knowledge in realistic situations rather than encountering policies only as abstract information. Nova uses customer conversations, troubleshooting problems, and escalation decisions to create practice that resembles the performance expected on the job.
Applied in Nova: Customer role-play, troubleshooting, and policy-based scenarios
Corrective & AI-Supported Feedback
Nova allows the learner to attempt a response before receiving targeted coaching. Feedback focuses on meaningful performance issues such as policy violations, security risks, incorrect escalation, skipped required actions, and unsupported promises.
Applied in Nova: Adaptive coaching, corrective feedback, and Meets the Standard logic
Reflection & Self-Regulated Learning
Learning does not end with receiving an answer. Nova encourages learners to evaluate responses, revise meaningful errors, use feedback, and become increasingly independent in their decision-making.
Applied in Nova: Coaching, reflection, retry decisions, and learner independence
Transfer & Performance Support
The purpose of training is successful performance beyond the learning environment. Nova combines practice with just-in-time access to approved procedures and escalation guidance rather than expecting employees to memorize every rule during onboarding.
Applied in Nova: Just-in-time procedural guidance and workplace decision support
How these ideas shape Nova
Learners practice realistic work tasks, respond before receiving coaching, get more practice when important errors remain, and can look up approved procedures while working. Nova 2.0 extends this into evaluation by showing how performance patterns could point to training gaps and inform later design decisions.
Connection to Graduate Study
Nova also extends ideas explored during my M.S. in Instructional Design and Technology at the University of Memphis. Graduate work in Artificial Intelligence and Learning examined AI-supported customer-service training using simulations, feedback, performance analytics, reflection, and workplace transfer. Nova develops many of those concepts into a functioning portfolio prototype.
Responsible AI
Responsible AI Design
A major design goal was reducing the risk of unsupported or fabricated company information. Nova is instructed not to invent ResolveIQ policies, procedures, benefits, timelines, or outcomes.
If the approved knowledge sources do not contain the requested information, Nova responds with a controlled fallback rather than generating an unsupported answer.
Example
Learner: “How many paid vacation days do ResolveIQ employees receive?”
Nova: “I can’t find that information in the approved ResolveIQ support resources. Please check with your manager or the appropriate ResolveIQ support team before taking action.”
Design principle: Accuracy is more important than answering every question.
Coaching logic
Performance-Based Coaching
Nova distinguishes between critical performance errors and minor opportunities for refinement.
Requires Another Attempt
- Violates an approved policy
- Creates a security or accuracy risk
- Uses the wrong escalation path
- Skips a required action
- Makes an unsupported promise
Meets the Standard
- Critical requirements are satisfied
- Response is safe and accurate
- Correct procedure is followed
- Appropriate escalation is used
When a learner meets the critical requirements, Nova marks the response Meets the Standard and may provide an optional refinement without forcing another retry.
Prototype analytics • Simulated ResolveIQ deployment
Nova 2.0 — Learning Analytics
Nova 2.0 explores what happens after learners interact with an AI performance-support tool. In a production deployment, conversation data could be classified by policy area, performance outcome, critical error, and training need—giving instructional designers a new source of evidence for continuous improvement.
The dashboard below uses a simulated dataset of 40 fictional ResolveIQ learner interactions to demonstrate this analytics layer.
40 — Simulated Learner InteractionsAcross six ResolveIQ performance areas
69% — Met the Standard25 of 36 assessed interactions
56% — Account Security Retry RateMost difficult area in the simulated deployment
4 — Wrong EscalationMost Common Critical ErrorObserved across multiple policy areas
Learner Outcomes
- Meets Standard25
- Needs Retry11
- Information Only4
Among interactions that evaluated learner performance, approximately 69% met the required standard.
Interaction Mix
- Scenario Practice20
- Procedure Help8
- Coaching6
- Quiz6
Scenario Practice accounted for half of all simulated interactions, reinforcing Nova’s role as a practice-and-performance tool rather than simply an FAQ chatbot.
Where Learners Struggled
- Account Security56%
- Missing Data & Recovery29%
- Billing & Refunds25%
- Technical Troubleshooting20%
- Account Access17%
- Escalation & Service Standards0%
Key finding: Account Security produced the highest retry rate. Learners most often struggled to recognize when normal support should stop and a security escalation should begin.
Critical Errors Identified
- Wrong Escalation4
- Unsupported Promise3
- Security Risk3
- Skipped Required Action1
Most common issue: Learners sometimes understood the customer problem but selected the wrong escalation path.
From Data to Design Decisions
Finding
Account Security was the most difficult policy area.
Learners had the most trouble recognizing when routine troubleshooting should stop and an account-security escalation should begin.
Instructional Design Response
Add an account-security scenario to onboarding and create a one-page Security vs. Technical Support decision tree.
Finding
Wrong escalation was the most common critical error.
Several learners understood the customer concern but chose the wrong team or pathway for resolving it.
Instructional Design Response
Add cross-policy escalation practice to onboarding and create a concise reference map for Billing, Technical Support, and Account Security.
How Nova 2.0 Would Work in Production
Step 01Learner Interaction
Step 02Conversation Data
Step 03AI Performance Classification
Step 04Aggregated Learning Analytics
Step 05Instructional Design Action
In a production environment, Nova interactions could be analyzed for patterns such as policy area, learner outcome, critical errors, escalation decisions, knowledge gaps, and recurring support needs. Those patterns could help L&D teams determine where additional practice, job aids, course revisions, or performance support are needed.
Portfolio prototype: The live Nova chatbot is fully functional. The analytics shown here use simulated ResolveIQ interaction data to demonstrate the proposed evaluation layer.
Analytics Framework
Interaction Type
Scenario Practice • Procedure Help • Coaching • Quiz
Policy Area
Account Security • Billing • Missing Data • Account Access • Technical Support • Service Standards
Learner Outcome
Meets Standard • Needs Retry • Information Only
Critical Error
Wrong Escalation • Unsupported Promise • Security Risk • Skipped Required Action
Training Gap
Policy Knowledge • Procedure Sequence • Escalation Decision • Security • Expectation Setting • De-escalation
Knowledge Gap
Flags questions that cannot be answered from approved ResolveIQ resources
Why these analytics matter
Analytics are useful only when they lead to action. Nova 2.0 demonstrates how conversational learning data could move beyond usage statistics and help instructional designers identify specific performance gaps, prioritize revisions, and make evidence-informed design decisions.
Contribution
My Role
- Instructional Design
- Performance Support Design
- Scenario Design
- AI Prompt Design
- Knowledge Architecture
- Corrective Feedback Design
- Responsible AI / Safe-use design
- User Experience Design
- Prototype Development
- Quality Assurance Testing
- Learning Analytics
- Evaluation Strategy
- Data-Informed Design
Tools & Technologies
- Zapier Chatbots
- Generative AI
- Lovable
- Knowledge-Based AI Retrieval
- Web-Based Performance Support
Reflection
Design Reflection
This project looks at how generative AI can extend learning beyond a traditional course. Instead of asking employees to memorize every procedure during onboarding, Nova gives them help when they need it along with realistic practice and feedback.
The project was a good reminder that the goal is not to deliver more content. The goal is to help people do the work.
Nova 2.0 extended the project from performance support into evaluation. Designing the simulated analytics layer demonstrated how conversational learning data could be used to identify recurring performance gaps, evaluate where learners struggle, and connect those findings to specific instructional-design responses. This reinforced the importance of designing not only for learning and performance, but also for continuous improvement.