ISO 9001 Analysis and Evaluation - DOs & DON'Ts

Practical advice on implementing ISO 9001:2015 Clause 9.1.3

This article examines a subclause in ISO 9001:2015, Performance Evaluation, namely 9.1.3, Analysis & Evaluation.

Peter Drucker, the management guru (allegedly) said:  'If you can't measure it, you can't manage it'. And this certainly applies to Management.  The reality is that many Quality Managers, or equivalent, ignore the requirements of clause 9.1.3.  To put time and effort into a management system and not know whether it benefits your organization can only be described as a stupid error. 

ISO 9001 Certification should be much more than a logo on your website. Performance evaluation is where you start.

NOTE: While the exact wording may vary across Standards, the advice provided here also applies to ISO 14001, ISO 27001, ISO 45001, and other Standards with the same HLS structure as ISO 9001.

Table of Contents

• Purpose of ISO 9001:2015 Clause 9.1.3
• This is the most likely source of a Major Noncompliance
• Sources of Data for Analysis and Evaluation
• Documenting Output of Analysis and Evaluation
• How to Implement Analysis and Evaluation for ISO 9001
  
  • DO's
    • Do ensure that the output from analysis and evaluation is in a suitable format
    • Do determine the appropriate frequency for evaluating and analyzing the information
    • Do make every effort to retrieve information electronically
    • Do ensure that the methods and data quality provide useful information
    • Do use statistical techniques for the analysis and evaluation processes
  • DON'Ts
    • Don't forget to evaluate the information you have decided to monitor
    • Don't continue to collect and evaluate information if no use is being made of it
    • Don't forget that there are seven headings in this sub-clause where analysis and evaluation are required
• Conclusion

Purpose of ISO 9001:2015 Clause 9.1.3

The intent of ISO 9001:2015 sub-clause 9.1.3 is for the organization to analyze and evaluate data and information from monitoring and measurement results.  It was formulated to determine:

• If processes, products, and services meet requirements,
 
• If corrections and corrective actions are needed,
 
• If decisions based on risk-based thinking were satisfactory, and
 
• Opportunities for improvement.

Evaluation is a requirement added in the 2015 version of the Standard. So, expect Auditors to have it on their ISO 9001 Audit Checklist.  ISO 9001 Consultants and QHSE Managers take note.

This is the most likely source of a Major Noncompliance

Having worked with this Standard for the past nine years and undertaken 100s of CAB audits, I can say that this area is often found to be weak and often undertaken in a meaningless way. 

Do not expect External Auditors to be anything other than conscientious in pursuing evidence of compliance with the requirements here. 

Just think about it for a moment.  Your organization has customers whom it supplies with products and services.  This is done in accordance with agreed contracts and/or specifications.  You also have objectives to improve your Quality Management System.  I could also add many more measurable features to your QMS.

Hertzberg promoted the idea that if you can't measure it, you can't manage it.  So, it makes no sense not to diligently measure and monitor the performance of your QMS or subsequently analyze and evaluate results. 

Top management requires these results at Management Reviews to have an evidence-based discussion of QMS effectiveness.

Sources of Data for Analysis and Evaluation

The selected data should focus on providing results to evaluate the performance and effectiveness of the quality management system and to determine the need for any improvements. 

Here are seven examples of typical data sources:

• Product: yield; conformity to specific requirements (e.g., customer, statutory, regulatory); rates of non-conformities [e.g., parts per million (ppm)]; scrap and rework; on-time delivery; fulfillment of an order;
 
• Service performance: queuing times; an indication of resolution of customer issues; ease of access; cleanliness; housekeeping; friendliness;
 
• Results from the monitoring of customer perception;
 
• Delivery of projects to plan (e.g., budget and timing);
 
• Review of action items on risks and opportunities (e.g., meeting minutes);
 
• On-time delivery and quality (e.g., rejects) for external providers;
 
• Status of quality objectives.

Documenting Output of Analysis and Evaluation

Typically, output from analysis and evaluation is documented as follows:

• Trend analyses or reports,
 
• Balanced scorecards and
 
• Dashboards,

And becomes an input to management reviews or meetings that consider the output.

How to Implement Analysis and Evaluation for ISO 9001

DOs

• Do ensure that the output from analysis and evaluation is in a suitable format.  It should allow a determination of whether actions are needed to improve the quality management system – this is of prime importance here.
 
• Do determine the appropriate frequency for evaluating and analyzing the information.  Information can be analyzed daily, weekly, monthly, or annually, depending on how you will use it.  
  
  For example, if you are currently experiencing daily issues with on-time deliveries, it makes no sense to evaluate performance monthly, as you need information on a daily or weekly basis to take timely action or to get timely feedback on actions already initiated to address the problem.
   
• Do make every effort to retrieve information electronically.  Avoid manual data preparation when possible.
 
• Do ensure that the methods and data quality provide useful information.  These data should be representative, unbiased, complete, and accurate.  Best of all, they should be usable for managerial decisions.
 
• Do use statistical techniques for analysis and evaluation.  Check-out Minitab.

DON'Ts

• Don't forget to evaluate the information you have decided to monitor.  Analyzing a lot of raw data is insufficient to meet the requirements.  You are advised to interpret the term 'valuation' as the conversion of the analyzed data into a format that facilitates decision-making and follow-up actions by management or other interested parties.

• Don't continue collecting and evaluating information if it isn't being used.  If no decisions and/or actions arise from the evaluated information, drop that data source from your program.  And never collect and evaluate data to benefit your CAB (Certification Body) alone.

 
• Don't forget that this subclause has seven headings that require analysis and evaluation.  Your CAB Auditor will be asking for evidence that you have addressed all seven – a list like this in a Standard constitutes "low-hanging fruit’' for an Auditor.

Conclusion

ISO Sub-clause 9.1.3 is a new clause for a reason.  Too often in the past, very low-grade information was developed in QMSs. This was of little use to top management in making useful decisions about the status and future development of their QMS. 

The expectation is that the analyzed data will be evaluated to facilitate decisions and actions regarding the QMS.  Your ISO 9001 Certification Auditor will seek this linkage.

NOTE:

1. This is part of an ongoing series of posts on the DOs and DON'Ts of implementing and maintaining a QMS.  Use the "search'' field at the top of this page to find the others.
2. Reference: ISO/TS 9002:2016, Guidelines for the application of ISO 9001:2015

Related Courses

Related Articles

• ISO 9001 Clauses
• ISO 9001 Documentation: DO's and DON'Ts 
• ISO 9001 Continual Improvement: DO's and Don'ts
• ISO 9001 Context of the Organization (COTO): DOs and Dont's
• ISO 9001 Knowledge Management: DOs and Don'ts
• ISO 9001 Planning for Changes: DOs and DON'Ts
  
  

deGRANDSON Global is an ISO Certified Educational Organization

We have chosen ISO 21001 certification because, unlike IRCA and Exemplar badges (which, in our opinion, are commercially compromised), it is based on independent third-party assessment.  It is a "university-grade" standard used by schools, colleges, and universities to demonstrate competence.

We offer Certified Courses for ISO 9001, ISO 13485, ISO 14001, ISO 17025, ISO 27001, ISO 45001, Data Protection, and Risk Management.