Layout 1


Plan Experiments to Prevent Problems

by Scott A. Laman

This article was featured in January 2016’s Best Of Back to Basics edition.

An experimenter’s ability to minimize mistakes often comes with experience. Unfortunately, this experience often results from past problems, surprises or confusion. After conducting many experiments—several at remote sites—I’ve seen critical details of experimentation go wrong.

A specific example was when I arrived at a production facility only to find that the equipment intended for the experiment had been replaced, and the operator who normally ran the process was unavailable.

Looking at many common mistakes in the same categories as used in the traditional fishbone diagram—manpower, machines, methods, materials, measurements and environment—can help experimenters plan.

The questions listed below should be answered affirmatively before conducting an experiment in order to minimize or avoid problems.


For an experiment to be successful, all personnel involved should be adequately trained and communication should be ongoing. Involving operators is also essential, as those performing the experiment provide valuable insights that cannot be obtained elsewhere. Planners need to ask:

  • Are the people sponsoring, planning and conducting the experiment trained in design of experiments?
  • Were operators involved in the planning?
  • Has a plan been made for communication between all shifts?


When considering machines, it is critical they be proven, in top condition and stable. As a special consideration for continuous processes, the time between runs should be long enough for the process to stabilize at each set of conditions. Experimenters should consider:

  • Have recent equipment changes been thoroughly documented?
  • Are the machine and equipment in the best possible condition?
  • If using a continuous process, has the transition time between runs been evaluated and determined to be long enough?
  • Has the process been stable enough to risk investing in an experiment?


Documentation tools such as experiment goal statements, written procedures and data recording forms fall under the methods category.

Key techniques, including randomization and a team based approach to variable selection, can apply. Experimental designs like factorial experiments can also be included in planning. Safety considerations and foreseeing the type of analysis also pay dividends. The questions to ask are:

  • Has a problem or goal statement been written, and has the purpose been defined?
  • Does the experiment completely address the problem statement while distinguishing between important main effects and interactions?
  • Have the appropriate responses and their limits been chosen?
  • Have quality or engineering tools been used for guidance?
  • Did a team select the process variables for study?
  • Have noise factors been identified and plans made for recording or control?
  • Have specific procedures been written for conducting the experiment?
  • Have the runs been randomized as much as possible, and has blocking been considered?
  • Are all the combinations of process conditions safe for personnel and equipment?
  • Is there a results analysis plan?


No matter what the industry or form of raw material, it is important to conduct the experiment with a representative lot. If at all possible, include those material characteristics in the experimental design. Otherwise, the results will not extend to the future process. Planners should consider:

  • Is the material typical of the process?
  • Can the results be extended to other lots of material?


Gauges and measurement devices must function as intended, with all calibrations completed. Procedures for data collection should be written, and the operators should be asked to record any other important information. After the experiment, the results should be verified. Questions to be asked include:

  • Have all gauges and measurement devices been checked and found to be in good working condition?
  • Have specific procedures been written for data collection?
  • Are important data being collected that are not part of the experiment?
  • Is there a follow-up plan to validate the results?


Environmental factors should be identified and evaluated for their potential impact on the experiment. At a minimum, a plan should be made for monitoring them. If necessary, variables such as ambient temperature, humidity, vibration and noise can be part of the experimental design. Questions to ask:

  • Have environmental factors been evaluated?
  • Has a plan been made for monitoring those factors?

Advanced training and experience of personnel leading experimentation is certainly important. However, many of the pitfalls—however simple—can be avoided by systematically relating potential problems to the logical categories of the fishbone diagram.


An example of a traditional fishbone diagram using these categories can be found at www. asq.org/qualityprogress.

SCOTT A. LAMAN is the manager of quality engineering for Arrow International Inc. in Reading, PA. He earned a master’s degree in chemical engineering from Syracuse University. He is a senior member of ASQ and is certified as a quality engineer, reliability engineer, quality manager, Six Sigma Black Belt and quality auditor.

Average Rating


Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers