The QA Playbook: Moving from 2% sampling to total coverage

Modern contact center Quality Assurance (QA) is the systematic process of evaluating customer-agent interactions to ensure they meet operational standards, compliance requirements, and customer expectations. Unlike legacy programs that rely on supervisors manually listening to a tiny fraction of calls, a modern QA program uses automated conversation intelligence to analyze every interaction, providing a complete view of floor performance.

Key takeaways:

• Shift from compliance to outcomes: Modern scorecards prioritize whether the customer’s problem was solved over whether the agent followed a rigid script.
• Automate the baseline: Use AI to handle objective checks (e.g., identity verification, disclosures) so human auditors can focus on nuanced coaching.
• Close the loop daily: QA data is only valuable if it reaches the agent within 24-48 hours of the interaction.
• Solve the sampling bias: Moving from 2% manual sampling to 100% automated coverage eliminates the risk of missing critical compliance failures or high-value customer insights.

Why the 2% sampling model is failing the floor

For decades, the industry standard for QA has been the "random sample." A supervisor or QA lead selects 3 to 5 calls per agent per month and scores them against a rubric. However, this method is statistically insignificant. If an agent handles 1,000 calls a month, a 5-call sample represents 0.5% of their work. This creates a high probability of "sampling bias," where an agent is penalized for one bad interaction that was an outlier, or conversely, a low-performer slides under the radar because their five sampled calls happened to be routine.

Furthermore, manual sampling often misses the "needles in the haystack"—the specific compliance violations or high-intent sales signals that occur in the other 98% of conversations. According to Gartner’s Customer Service & Support practice, the focus for 2026 is shifting toward domain-specific AI and data protection to address these visibility gaps. For operations leads, the goal is no longer just to score calls; it is to gain a total data set of what is happening on the floor.

Phase 1: Designing the outcome-based scorecard

A modern scorecard should be split into three distinct categories: Mandatory Compliance, Process Accuracy, and Soft Skills/Experience.

1. Mandatory Compliance (Pass/Fail): These are non-negotiable items like PCI compliance (not recording credit card numbers), identity verification, and required legal disclosures.
2. Process Accuracy: Did the agent follow the correct workflow in the CRM, such as Salesforce Service Cloud or Zendesk? This measures if the technical resolution was correct.
3. Experience & Soft Skills: This measures empathy, active listening, and professional tone.

Instead of asking "Did the agent use the customer's name three times?", modern scorecards ask "Did the agent acknowledge the customer's frustration?" and "Was the resolution confirmed by the customer?" Shifting toward these qualitative outcomes makes the QA process feel more like support and less like a police action to the agents.

Phase 2: Building the tech stack for full coverage

To move beyond sampling, you need an integrated stack that connects your telephony to an intelligence layer. Most modern contact centers start with a CCaaS platform like Genesys, Five9, or Talkdesk. These platforms provide the raw audio and metadata for every call.

However, the CCaaS alone usually doesn't provide deep QA automation. This is where a conversation intelligence layer is required. For example, teams often pair a CCaaS platform with a tool like Hear.ai (powers our network) to analyze customer conversations at scale. This allows the QA team to gain coverage across all calls rather than just samples, automatically flagging compliance risks or specific keywords that indicate a frustrated customer.

By using infrastructure from Google Cloud or AWS to process these transcripts, operations leads can search across thousands of hours of audio for specific trends, such as how often a new competitor is mentioned or where a new promotion is causing confusion.

Phase 3: Automating the objective, humanizing the subjective

The most efficient QA programs use a "Hybrid Scoring" model.

What to automate: AI models are excellent at checking for specific phrases or the absence of them. You can automate the scoring for "Opening Greeting," "Verification," and "Mandatory Disclosure." This frees up your QA specialists from the drudgery of checking boxes.

What to keep human: Humans are still better at judging nuance, sarcasm, and complex problem-solving. Use your QA staff to review the calls that the AI flags as "High Emotion" or "Atypical Resolution." This ensures that your most expensive resources (your people) are focused on the most complex interactions.

Metrigy research into CX/AI success metrics suggests that companies integrating AI into their QA workflows see higher agent satisfaction because the feedback is perceived as more objective and comprehensive.

Phase 4: Calibration and the feedback loop

QA is useless if it is not calibrated. Calibration is the process where multiple auditors score the same call to ensure they reach the same conclusion. In a modern program, this should also include "AI calibration"—testing the automated system against human judgment to ensure the sentiment analysis or keyword flagging is accurate.

Once a score is finalized, the delivery is critical. Feedback should be:

• Timely: Within 48 hours of the call.
• Specific: Linked to a specific timestamp in the recording.
• Actionable: Include a specific behavior to change or a resource to review.

Integrating these scores back into a workforce management (WFM) or coaching platform ensures that supervisors are spending their 1-on-1 time on the right behaviors. If the data shows an agent is consistently failing on "Resolution Confirmation," the supervisor doesn't need to guess what to talk about during their weekly meeting.

How to measure QA program success

Avoid the trap of measuring the QA team by "number of calls scored." This is a volume metric that doesn't track value. Instead, track:

• QA Correlation to CSAT/NPS: Are agents with high QA scores actually delivering better customer satisfaction? If not, your scorecard is measuring the wrong things.
• Coaching Impact: Does an agent's score in a specific category improve in the two weeks following a coaching session?
• Compliance Gap Reduction: Are the number of automated "compliance flags" trending down over time?

Forrester’s CX Index tracks how customers rate their experiences across brands; a well-run QA program should be the internal leading indicator of where your brand will land on that index.

FAQ

How many QA evaluators do I need per agent? In a manual model, the ratio is often 1:50 or 1:100. However, with automation tools like Hear.ai handling the baseline scoring, a single QA lead can often oversee 200+ agents by focusing only on outliers and high-risk interactions.

Should agents be able to dispute a QA score? Yes. A formal dispute process is essential for trust. If an agent feels a score is unfair, they should be able to flag it for a second review. In a 100% coverage model, disputes actually decrease because the agent knows the score is based on a total body of work, not a single "unlucky" call.

Can we use QA scores for compensation? Many centers do, but it should be a balanced metric. Using QA scores as the sole driver of bonuses can lead to "gaming the system." It is more effective to use QA as a gatekeeper for performance tiers rather than a direct dollar-for-point calculation.

What is the difference between QA and Conversation Intelligence? QA is the process and the rubric; Conversation Intelligence is the technology that enables it. You can have QA without CI (manual), but you cannot have modern, 100% coverage QA without a CI layer to process the data.

By moving from a reactive, sampling-based approach to a proactive, full-coverage strategy, contact center leaders can turn QA from a back-office cost center into a primary driver of operational intelligence. For more on optimizing your floor, see our guides on [measuring-agent-performance.html] and [coaching-for-retention.html].