Assess Your Lifetime Model
A graphical distribution fit test for product life data analysis
by Necip Doganaksoy, Gerald J. Hahn and William Q. Meeker
Analysis of product lifetime data typically calls for estimating product reliability—together with an associated statistical confidence interval—after a specified lifetime. For product lifetimes that do not exceed the longest time to failure in the given data, such estimates and intervals can be calculated without making any assumption about the statistical lifetime distribution, using so-called “nonparametric” methods.
Reliability estimates, however, are frequently desired for lifetimes that exceed—sometimes by far—the longest observed time to failure. For example, you might wish to estimate five-year reliability from (up to) two-year product lifetime data. This requires you to assume an appropriate lifetime distribution. But what distribution should you select and how can you assess the wisdom of your choice?
Physical, as well as statistical considerations are factored into this determination (see sidebar, “Considerations in Identifying a Suitable Lifetime Distribution”). In some situations, previous experience with similar products might suggest a distribution. In addition, however, you must assess the suitability of the assumed distribution from the data and potentially consider alternatives. This is easily done by comparing probability plots of the data under different distributional assumptions.
This column describes, via two examples, a graphical method to help the analyst assess the appropriateness of an assumed lifetime distribution—technically referred to as a statistical “goodness of fit” evaluation—through visual examination of probability plots.
Considerations in Identifying a Suitable Lifetime Distribution
The Weibull and the lognormal distributions are the most popular statistical distributions used in product life data analysis.
Sometimes, you can identify a suitable distribution based on your understanding of the failure mechanism of the product. For example, it can be shown that if failure takes place at the occurrence of the first of several independent and identically distributed failure modes (for example, the weakest link in a chain), the distribution of the resulting lifetimes can be approximated by a Weibull distribution. In fact, the Weibull distribution has, on occasion, been found to provide a good fit to the lifetimes of various products, including insulation materials, steel bars, x-ray tubes, ball bearings, capacitors and ceramic parts. Moreover, a Weibull distribution can represent lifetimes of products with increasing, decreasing or constant hazard functions over time. (The hazard function characterizes the propensity of a unit to fail in the next small interval of time, conditional on having survived a specified time. A Weibull distribution shape parameter less than/exceeding one is indicative of a decreasing/increasing hazard function).
The exponential distribution is a special case of a Weibull distribution with shape parameter one. It has the important property that its hazard function is constant over time. This distribution has been used to model the lifetimes of electronic components which do not physically degrade during their useful life and whose failures are attributable to some random external event, such as being struck by lightning.
The lognormal distribution has been justified theoretically as a model for lifetime data by a so-called “central limit” theorem. This states that the distribution of a random variable that arises from the product of several positive random quantities, none of which dominate, can be described by a lognormal distribution. Thus, lifetimes for products whose failure modes arise from an accumulation of damage over time are often described well by a lognormal distribution. This distribution also is used often as an approximate model for electronic components.
—N.D., G.J.H. and W.Q.M.
Various other distributions for product life and their applications are described in the following sources:
Meeker, William Q., and Luis A. Escobar, Statistical Methods for Reliability Data, John Wiley & Sons Inc., 1998, chapters 4 and 5.
Nelson, Wayne, Applied Life Data Analysis. John Wiley & Sons Inc., 1982, chapter 2.
Example No. 1: brake assembly defect
This example is based on one of our previous QP columns.1 A manufacturer of home appliances identified a defect in the brake assembly of a washing machine. Investigations showed that 287,091 appliances were built and shipped with the faulty assembly and 187 of these units failed at various recorded times during their first year in the field. The manufacturer needed a quantitative assessment of product reliability over a three-year warranty period and also at various times after the warranty period had ended, based on statistical analysis of the available field data.
Figure 1 displays Weibull and lognormal distribution probability plots for the one-year brake assembly lifetime data. The straight line is the maximum likelihood estimate of the assumed lifetime distribution (see sidebar “A Refresher on Probability Plots and Maximum Likelihood”). The 187 failure times were rounded to the midpoint of the month during which the failure occurred and, as a result, the probability plots show only 12 points. For both assumed distributions, the points scatter closely around their fitted straight lines, suggesting that the Weibull and the lognormal distributions provide a good fit within the data range. However, a more formal assessment of the statistical goodness of fit of the two alternative lifetime distributions is desired.
A Refresher on Probability Plots and Maximum Likelihood
A probability plot for an assumed lifetime distribution model is a plot of the estimated fraction failing as a function of time on special nonlinear axes that are constructed so that the plotted points tend to scatter around a straight line if the assumed distribution is correct. For censored lifetime data (that is, data with one or more unfailed units whose lifetimes to date are known), only the lifetimes of the failed units are plotted, but the censoring is considered in determining the position at which these points are plotted. The straight line around which the points are supposed to scatter provides statistical estimates of the failure probability as a function of time (or the lifetime cumulative distribution function). These estimates are typically obtained by the method of maximum likelihood (ML). (The basic principle underlying ML is to choose as estimates those values of the distribution parameters—from among all possible parameter values—that make the observed data most likely).
—N.D., G.J.H. and W.Q.M.
A graphical goodness-of-fit test
Probability plots can be made more informative by plotting nonparametric simultaneous confidence bands on the failure probabilities as a function of time. (Simultaneous means that the coverage probability of the confidence bands apply over the entire range of failure times, as displayed on the plot, rather than for a specified single time). Figure 2 shows Weibull and lognormal distribution probability plots of the brake assembly data, along with the straight line showing the failure probability ML estimates and approximate 95% nonparametric simultaneous confidence bands. In particular, the bands displayed in Figure 2 provide confidence intervals for the proportion failing as a function of months in service up to the lifetime of the failed unit with the longest lifetime (that is, 12 months).
In addition, such confidence bands can be used to formally assess the appropriateness of the assumed distribution. Specifically, Vijayan N. Nair showed that if you can draw a straight line within the area of the calculated nonparametric simultaneous confidence band, you can conclude that the data are consistent with the distribution assumed by the probability scale of the plot, at least within the range of the data.2 If, on the other hand, you cannot draw a straight line within the nonparametric simultaneous confidence band, there is statistical evidence that the lifetimes did not come from the distribution used to construct the probability plot.
For the brake assembly example, it is evident from examination of Figure 2 that a straight line can be easily drawn to pass within the nonparametric simultaneous confidence band for the Weibull and the lognormal distributions. Therefore, you can now formally conclude that there is no statistical evidence to contradict the assumption of either a Weibull or of a lognormal distribution for product lifetime within the range of the data.
This example is not unusual. Often, the data do not provide adequate information to discriminate between alternative distributions, especially with small sample sizes or a small number failing. If two distributions appear to provide equally good fits to the data, as judged by the scatter of points around the fitted line, you might analyze the data under both assumptions and compare the results.
As discussed in our 2010 QP column, “Predicting Problems,”3 in extrapolating beyond about 15 months in this example, the Weibull and lognormal reliability estimates tend to diverge, with the predictions using the Weibull distribution being more pessimistic than those based on the lognormal distribution.
Thus, because it is more conservative, the fitted Weibull distribution was used for predictions. Also, you must keep in mind the critical assumption that a statistical distribution that provides an adequate fit to the data within the range of the data (first year of life in the example) also applies when extrapolated (3 three-year life). This assumption may not be true and when it is not, the predictions can be far from accurate. The prediction intervals do not reflect possible deviations from this assumption.
Example No. 2: glass tensile strengths
This example, deals with tensile strength testing of glass specimens.4 This test characterizes the maximum stress that the specimens can withstand under a predefined load before breaking. Because the glass specimens are inexpensive, and life testing them is simple, large sample sizes may be encountered. In this application, a random sample of 150 glass specimens built using a new furnace were subjected to tensile strength testing.
For purposes of illustration, the lifetime sample in this example was created by randomly generating, using computer simulation, 150 observations from a lognormal distribution with a shape parameter of 0.25 and median of 35 MPa. This example did not involve any censoring; all 150 observations were taken to be lifetimes.
Figure 3 displays the Weibull and lognormal distribution probability plots of the data, along with approximate 95% nonparametric simultaneous confidence bands. Not surprisingly because the data were generated from a lognormal distribution, this distribution appears to provide a good fit to the data. This is reinforced by the fact that you can draw a straight line within the simultaneous nonparametric confidence band on the lognormal distribution probability plot. On the other hand, the Weibull distribution is determined to be an inappropriate model for the data because you cannot draw a straight line through the simultaneous nonparametric confidence band on the Weibull distribution probability plot.
The procedure described here offers an appealing visual alternative to conventional statistical goodness-of-fit procedures that output a p-value, on which a decision to reject the proposed distribution or not is made.
To learn more: The simultaneous confidence bands in Figures 2 and 3 are based on the equal precision method presented in Vijayan N. Nair’s previous work.5-6 William Q. Meeker, and Luis A. Escobar7 show how these bands are calculated.
Software: Probability plotting of censored life data is now readily accessible in many statistical software packages. However, the nonparametric simultaneous confidence bands described in this column are not, as yet, widely available. In the illustrations of this column, we used the JMP software to calculate and display such confidence bands. The km.ci package of the R language may also be used to obtain such bands.
Cautionary note: Similar to formal goodness-of-fit tests, with a small amount of data, an assumed distribution will generally not be rejected using the method we have described, even though it might be a seriously inadequate model. On the other hand, with a large amount of data, the test for any distribution is likely to be rejected, even when the assumed distribution provides an excellent approximation of the true model. Thus, a plot showing the nonparametric estimate and the associated simultaneous confidence band has the advantage of providing an assessment of practical and statistical significance.
Note that statistical analysis can take you just so far. For the data analysis to really be meaningful, it must be accompanied by an understanding of the underlying physical situation. In product life data analysis, this includes, for example, an appreciation of the impact on the product lifetime distribution of multiple failure modes.8
This is especially critical when extrapolation beyond the lifetimes of all of the failed units is required. As illustrated by our Example 1, when a distribution provides a good fit to your data, it does not imply that that the chosen distribution will provide valid inferences when extrapolating outside the range of the data.
References and Note
- William Q. Meeker, Necip Doganaksoy and Gerald J. Hahn, “Predicting Problems: Forecasting the Number of Future Field Failures,” Quality Progress, November 2010, pp. 52-55.
- Vijayan N. Nair, “Confidence Bands for Survival Functions With Censored Data,” Biometrika, 1981, Vol. 68, pp. 99-103.
- Meeker, “Predicting Problems: Forecasting the Number of Future Field Failures,” see reference 1.
- The example was adapted from Necip Doganaksoy, Gerald J. Hahn and William Q. Meeker’s “Fallacies of Statistical Significance,” Quality Progress, November 2017, pp. 56-62.
- Nair, “Confidence Bands for Survival Functions With Censored Data,” see reference 2.
- Vijayan N. Nair, “Confidence Bands for Survival Functions with Censored Data: A Comparative Study,” Technometrics, 1984, Vol. 26, pp. 265-275.
- William Q. Meeker and Luis A. Escobar, Statistical Methods for Reliability Data, John Wiley & Sons Inc., 1998, chapters 3 and 6.
- Necip Doganaksoy, Gerald J. Hahn and William Q. Meeker, “Reliability Analysis by Failure Mode,” Quality Progress, June 2002, pp. 47-52.
Necip Doganaksoy is the Douglas T. Hickey chair in business and associate professor at Siena College School of Business in Loudonville, NY, following a 26-year career in industry, mostly at General Electric (GE). He has a doctorate in administrative and engineering systems from Union College in Schenectady, NY. Doganaksoy is a fellow of ASQ and the American Statistical Association.
Gerald J. Hahn is a retired manager of statistics at the GE Global Research Center in Schenectady. He has a doctorate in statistics and operations research from Rensselaer Polytechnic Institute in Troy, NY. Hahn is a fellow of ASQ and the American Statistical Association.
William Q. Meeker is professor of statistics and distinguished professor of liberal arts and sciences at Iowa State University in Ames. He has a doctorate in administrative and engineering systems from Union College. Meeker is a fellow of ASQ and the American Statistical Association.