Sufficient Evidence

How to apply risk management to supplier evaluations

by John Suedbeck

This article was featured in January 2016’s Best Of Back to Basics edition.

Quality professionals make decisions based on evidence that is reliable, relevant and sufficient. The International Conference on Harmonization (ICH) of Technical Requirements for Registration of Pharmaceuticals for Human Use ICH Q9 guide1 provides guidance for applying quality risk management to supplier evaluation systems.

With regard to the consideration of sufficient audit evidence, it may be helpful to begin with the evaluation of a contract manufacturing organization (CMO). For many organizations, a CMO supplier may represent the greater risk and challenge for obtaining sufficient evidence. You begin managing risk by identifying and documenting your quality requirements and clearly communicating them to the supplier.

The senior management team may visit the CMO site to help communicate the requirements and obtain sufficient evidence to make a decision to move the process forward or stop it. The team must answer the question, "Is the supplier able to meet our quality requirements with regard to a high-level review of the project?"

Next, a technical team may visit the CMO site, performing tasks similar to the management team’s. The technical team’s questions, however, will be: "Are they able to meet our quality requirements for a more detailed review of the project, and do they have the technical ability to meet our quality requirements?"

After these visits, adequate project management can be the most powerful tool for managing risk associated with a contract manufacturer. Who should provide this talent? A strong argument can be made for the contract manufacturer.

The customer may provide details for project coordinators to ensure timely, efficient communication of information between the customer and supplier. After a project plan is generated, it should be reviewed by the auditor for adequacy, giving you more sufficient evidence for a decision about the supplier.

An on-site audit at this point may reveal evidence of adequacy, effectiveness and compliance. It could be argued that you’re still at the selection stage, and a sample, such as an exhibit batch, may be required to continue managing risk. An on-site audit following the exhibit batch could show sufficient evidence related to adequacy, effectiveness and compliance, and provide evidence of their overall ability to meet your quality requirements. This is another juncture at which to decide whether you should continue.

If you go forward with the supplier, risk is continually managed by data analysis and project management activities. Note that the focus for sufficient evidence has leaned heavily on whether the supplier can meet quality requirements. This same approach can be used for all of your suppliers in determining whether you have collected sufficient evidence.

Applying risk management to the supplier evaluation system can be better understood when you begin with the basics, collecting reliable, relevant and sufficient evidence.


  1. ICH Q9 is a U.S. Federal Drug Administration standard on quality risk management developed by the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use. For details, visit http://tinyurl.com/fdarisk.
  2. For more information on reliability and relevancy of audit evidence, read John Suedbeck, "Solid Proof," Quality Progress, June 2012, p. 72, and Suedbeck, "Bucket List," Quality Progress, January 2014, p. 64.

John Suedbeck is a quality assurance supervisor at Mayne Pharma in Greenville, NC. He earned a bachelor’s degree in chemistry from Fayetteville State University in North Carolina. An ASQ senior member, Suedbeck is an ASQ-certified quality auditor, improvement associate and manager of quality/organizational excellence.

In my own career as a computer components SQE I developed and used an audit checklist to identify risks. This created a profile of the prospective suppliers strengths and weaknesses. We then worked with the chosen supplier to strengthen their weaknesses. We also put in place incoming inspections and testing as contingency measures to protect our company from the suppliers weaknesses. The results from our inspections and testing were communicated back to the supplier to further drive their improvement efforts. They in turn communicated their testing results to help identify any test correlation issues. As our inspection and test data improved we reduced then dropped our incoming inspections. We then only monitored the final system test results and our customer feedback to verify acceptable quality and reliability. Lots of work but it resulted in developing low cost but high quality suppliers.
--Duane Briggs, 08-11-2015

Average Rating


Out of 1 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers