When organizations implement metrics, there is often the urge to track and trend any and every data point. The presence of a Quality Management System (QMS) makes this initiative significantly easier. When quality functions are managed within one system, extraction and analysis of data becomes less cumbersome, thereby saving time. If data is captured in the same manner, there is an apples-to-apples approach to the analysis of QMS metrics.  

Steps for Developing QMS Metrics

To develop QMS metrics for your leadership team, however, you must first gather and analyze information to determine what is pertinent to share. Below we outline three steps for developing QMS metrics to share with your leadership team: 

Step 1: Identify and Gather Metrics 

Identify the Key Performance Indicators (KPIs) needed for each process. You’ll want to establish metrics to track monthly and determine a target percentage for each. The target percentage should be established by considering the level of risk to patients. For example: 

Batch Record Review:  

• How many batch records are received/reviewed?  
• What type of errors are being found, if any?  
• What is the First-Time Right percentage?

On-Time Training Completion:

• Was training completed on or prior to the due date? 

Complaint Closure: 

• Were product quality complaints closed on or prior to the due date?  
• Was there a recurrence of the same product or lot number?  
• Was there a trend with respect to complaint type or lot number? 

Deviation Closure: 

• Were deviations closed on or prior to the due date? 
• Was there a recurrence of the same deviation type or root cause?  
• Was there a trend with respect to deviation type?  

Corrective and Preventive Action (CAPA) Closure:  

• Were CAPAs closed on or before the due date?  
• Were effectiveness checks required? If so, were they successful?  

Change Control Implementation: 

• Were change controls implanted on or before the target implementation and due date?  
• Were effectiveness checks required? If so, were they successful? 

Step 2: Track and Trend Data 

The key to sharing metrics with your leadership team is to first collect information and trend data. Start with gathering your established metrics monthly. It’s important to look at the change in QMS metrics over time (month to month). For example: Are you meeting your target percentages each month? If not, can you determine a root cause and corrective action? 

Once you have a year’s worth of data, you can transition to looking at changes quarterly rather than monthly.   

Step 3: Determine QMS Metrics to Share with Leadership Team 

Leadership teams are interested in streamlined, high-level metrics. Some things to consider are: Is your QMS functioning as it should? Are there any areas of concern? If your target percentages are not being met, what is the root cause, and how is this being addressed? 

An effective way to share metrics is via a Clustered Column Combination Chart. For Example: The chart below shows a typical example of On-Time Training Completion data for 2022. Immediately, we can see the data trend and whether the target percentage is being met. A key message should include clear and concise actions to improve the percentage and meet the target moving forward.   

*Target percentages shown are for illustration purposes only. 

Summary 

Following these three steps will assist in developing metrics that provide key information your leadership team needs to stay informed, also allowing them to take action and resolve any identified issues. Having a QMS system in place will provide clear evidence of a robust quality review process and continuous improvement. This will also aid in documentation that you are meeting regulatory requirements.  

Clarkston Consulting has the quality experience and resources to support you in the development of appropriate metrics for your organization. We can help you assess KPIs, develop analytics, and prepare information for the leadership team.