Last time I posted, I showed you how to divide a data set into training and validation samples in Minitab with the promise that next time I would show you a way to use the validation sample. Regression is a good analysis for this, because a validation data set can help you to verify that you’ve selected the best model. I’m going to use a hypothetical example so that you can see how it works when we really know the correct model to use. This will let me show you how Minitab 17’s Predict makes it easy to get the numbers that you need to evaluate your model with the training data set.

(The steps I used to set up the data appear at the end, if you want to follow along. If you do, consider skipping the steps where I set the base for the random numbers: If you produce different random numbers, the conclusion of the exercise will still be the same for almost everyone!)

Let’s say that we have some data where we know that Y = A + B + C + D + E + F + G. In regression, we usually  cannot measure or identify all of the predictor variables that influence a response variable. For example, we can make a good guess about the number of points a basketball player will score in his next game based on the player's historical performance, the opponent's quality, and various other factors. But it's impossible to account for every variable that affects the number of points scored every game. For our example, we’re going to assume that the data we can collect for prediction are only A, B, C, and D. The remaining predictors, E, F, and G are real variables, but they’re going to become part of the error variation in our analysis. E, F, and G are independent of the variables that we can include in the model.

Let’s say that we collect 500 data points and decide that we can use half to train the model and half to validate the model. Then we’ll do regression on the training sample to identify some models we think are the most like the real relationship. For clarity, I'll append _1 to the variable names when I'm using the training data set, and _2 to the names when I'm using the validation data set.

To start, I'll try fitting a model that has all the predictors that we can use in the training data set, and all of the interactions between those terms. Here's how to fit that model in Minitab 17:

1. Choose Stat > Regression > Regression > Fit Regression Model.
2. In Responses, enter 'Y_1'.
3. In Continuous predictors, enter 'A_1'-'D_1'.
4. Click Model.
5. Under Predictors, highlight 'A_1', 'B_1', 'C_1', and 'D_1'.
6. Under Add terms using selected predictors and model terms in Interactions through order, select 4. Click Add. Click OK twice.

Model Summary

      S       R-sq      R-sq(adj)  R-sq(pred)
1.76161  65.06%     62.82%      59.28%

This model should come close to maximizing the r2 statistic for this sample data. Once we have this model, Minitab 17 helps out a lot. We can quickly store the predictions from the validation data set to evaluate the model.

1. Choose Stat > Regression > Regression > Predict.
2. In the drop-down menu, select Enter columns of values.
3. In the table, enter the columns of predictors from the validation data set: 'A_2', 'B_2', 'C_2', and 'D_2'. Click OK.

The predictions for the model are now stored in the worksheet. Remember that we know that this model is wrong. No interaction effects are in the equation for the response that we defined as Y = A + B + C + D + E + F + G. Also, we know that the variables are unrelated so none of the interactions are related to the variables that we reserved for the error term.

One way to proceed is to remove terms from the model based on their statistical significance. For example, you might use the default settings with Minitab 17's stepwise selection procedure to find a new candidate model. Here's how to do that in Minitab 17:

1. Choose Stat > Regression > Regression > Fit Regression Model.
2. Click Stepwise.
3. In Method, select Stepwise. Click OK twice.

Model Summary

      S       R-sq     R-sq(adj)    R-sq(pred)
1.75428  64.02%     63.13%      61.98%

The new model has slightly higher adjusted and predicted r2 statistics than the previous model, so it is an acceptable candidate model. This reduced model still includes two interaction terms, A_1*C_1 and B_1*C_1. So we know that this model is also wrong because we know that the real relationship doesn't include any interactions. We'll store the predictions from this model using the same steps as we did for the previous model.

Let's also do a regression with the model that we know is most like the true relationship. Here's how to quickly get that model in Minitab 17:

1. Choose Stat > Regression > Regression > Fit Regression Model.
2. Click Stepwise.
3. In Method, select None. Click OK.
4. Click Model.
5. Click Default. Click OK twice.

Model Summary

      S       R-sq    R-sq(adj)    R-sq(pred)
1.77427  62.89%     62.28%      61.35%

Although we know that this model is the most true, the Model Summary statistics are worse than the statistics for the model that was the result of the stepwise selection. We might still use the principal of parsimony to prefer this model, but let's see what happens when we use the validation data.

Once you have the predictions stored from all 3 models, you can use different criteria to see which model fits the validation data the best, such as the predicted error sum of squares and the absolute deviations. One traditional criterion is the same one that we use to estimate the regression coefficients, minimizing the sum of the squared errors from the model. To do this in Minitab, do these steps for each model:

1. Choose Calc > Calculator.
2. In Store result in variable, enter an unused column.
3. In Expression, enter an expression like this one: sum(('Y_2'-‘PFITS1’)^2). (In this expression, Y_2 is the name of the response column from my validation data set and PFITS1 contains the predictions from the largest model.) Click OK.

If you calculate the sums for the three models above, you get these results:

Model

Sum of the squared errors

Full model, including up to the 4-way interaction. 

812.678

Stepwise model

787.359

Model with the terms from the real relationship

774.574

The conclusion of the analysis, which should happen, is that the best predictions come when you try to estimate the model that’s closest to the terms in the real relationship! Because Minitab’s new Predict lets you store the predicted values from a model, you can easily compare those predictions to the real values from a validation data set. Validation can help you have more confidence in your fearless data analysis.

Steps to set up data

1. Enter these column headers:

C1

C2

C3

C4

C5

C6

C7

C8

Y

A

B

C

D

E

F

G

2. In Minitab, choose Calc > Set Base.
3. In Set base of random generator to enter 1. Click OK.
4. Choose Calc > Random Data > Normal.
5. In Number of rows to generate, enter 500.
6. In Store in column(s), enter A-G. Click OK.
7. Choose Calc > Calculator.
8. In Store result in variable, enter ‘Y’.
9. In Expression, enter 'A'+'B'+'C'+'D'+'E'+'F'+'G'. Click OK.
10. Choose Calc > Make Patterned Data > Simple Set of Numbers.
11. In Store patterned data in, enter Samples.
12. In From first value, enter 1.
13. In To last value, enter 2.
14. In Number of times to list each value, enter 250. Click OK.
15. In Minitab, choose Calc > Set Base. Click OK.
16. Choose Calc > Random Data > Sample From Columns.
17. In Number of rows to sample, enter 500.
18. In From columns, enter Samples.
19. In Store samples in, enter Samples. Click OK.
20. Choose Data > Unstack Columns.
21. In Unstack the data in, enter Y-G.
22. In Using subscripts in, enter Samples.
23. Under Store unstacked data in, select After last column in use. Click OK.

We cannot improve what we cannot measure. Therefore, it is critical that we conduct a measurement systems analysis (MSA) before we start analyzing our data to make any kind of decisions.

When conducting an MSA for continuous measurements, we typically using a Gage R&R Study. And in these Gage R&R Studies, we look at output such as the percentage study variation (%Study Var, or %SV) and the Number of Distinct Categories (ndc) to assess whether our measurement system is adequate.

Looking at these 2 values to assess a measurement system often leads to questions like "Should I look at both values? Will both values simultaneously indicate if my measurement system is poor? Are these 2 values related?" 

The answer to all of these questions is "Yes," and here's why.

To clearly understand how number of distinct categories and percentage study variation are related, first consider how they are mathematically defined:

where sigma represents the square root of the variance components.

Using substitution, we can express the relationship between ndc and %SV as:

The last equation shows that ndc and %SV are inversely proportional: the larger %SV is, the smaller the ndc is, and vice-versa. However, it also suggests that the value of ndc depends not only on %SV, but on the variance components as well.

To simplify the equation and represent ndc solely as a function of %SV, we can express the variance components in another way. The total variance is the sum of two variance components, one corresponding to gage repeatability and reproducibility and the other to part-to-part variation:

Solving for sigma-squared for part and dividing each side of the equation by sigma-squared for total yields:

Because %SV / 100 = sigma gage / sigma total, the equation above can be rewritten as:

Substituting this value into the previous equation for ndc gives the following simplified formula:

This equation clearly shows the relationship between ndc and %SV and can be used to calculate the number of distinct categories for a given percentage study variation. As shown in Table 1, the calculated ndc value is then truncated to obtain a whole number (integer).

For example, if the calculated value is 15.8, mathematically you are not quite capable of differentiating between 16 categories. Therefore, Minitab Statistical Software is conservative and truncates the number of distinct categories to 15. For practical purposes, you can also round the calculated ndc values to obtain the number of distinct categories.

You can evaluate a measurement system by looking only at the number of distinct categories and using the following guidelines (based on the truncation method used by Minitab):

• ≥ 14 distinct categories – the measurement system is acceptable
• 4-13 distinct categories – the measurement system is marginally acceptable, depending on the importance of the application, cost of measurement device, cost of repair, and other factors
• ≤ 3 distinct categories – the measurement system is unacceptable and should be improv

These guidelines have some limitations. For example, in some cases when the %SV is over 30% the number of distinct categories is 4. Therefore, a measurement system with 32% study variation, which is unacceptable under the AIAG criteria for %SV, is acceptable under the ndc criteria. To avoid this discrepancy, some authors suggest only accepting a measurement system when it can distinguish between 5 or more categories. Although this fixes the original problem, it makes measurement systems with a 28-30% study variation unacceptable, because their corresponding ndc value equals 4.

To resolve this issue you can establish more specific guidelines based on the exact calculated ndc values, without truncating or rounding. For example, you could define an unacceptable measurement system based on an ndc < 4.497.

And that is how the number of distinct categories is related to %Study Var.

Many of my close friends and family members are nurses, and I’ve noticed (via Facebook of course) that many recently celebrated National Nurses’ Week, which ran Tuesday, May 6 through Monday, May 12—the actual birthday of Florence Nightingale.

Florence Nightingale is known as the founder of modern nursing, but you might not know that she is also a celebrated statistician. When I started reading more about her contributions to statistics, I was surprised to find that she used statistics to not only learn more about how many people died during wartime, and why, but also as a way to visually persuade people of the need for social change.

The article “Florence Nightingale: The passionate statistician” by Julie Rehmeyer goes into more detail, but I’ve compiled some information below on how Nightingale pioneered the use of statistics to improve hospital conditions, save lives, and develop policy for sanitation reform.

Nightingale was tasked to serve at a British hospital in Turkey during the Crimean War when reports got back to Britain about the horrific hospital conditions for the wounded there. She and a team of volunteer nurses addressed many issues, such as an overworked medical staff, neglected hygiene, and short supply of medicines and food, and by the time they left Turkey after the war ended in July 1856—the hospitals were improved and death rates were reduced significantly.

However, when Nightingale returned to Britain, she worried that what happened in Turkey would happen again if what was learned wasn’t acted upon in the future. She began to document her findings, including the medical data she collected while in Turkey, in a report that she hoped would help her to identify exactly why wartime hospital deaths were so high.  She found her report in need of more significant statistical analysis, and joined forces with William Farr, the inventor of the field of medical statistics.

With their analysis, Nightingale learned that lack of sanitation was the main cause of death during the Crimean War, not the short supply of medicines or lack of food. She also discovered that in peacetime, soldiers in England died at twice the rate of civilians—confirming that military health service inadequacies were causing more damage than was previously thought. Sanitation reform was badly needed.

Rather than presenting a long report with only words and tables of numbers to the Queen, Nightingale presented her findings graphically, which she thought would allow the Queen and her council to better understand the statistics. Known as a “coxcomb,” Nightingale used a structure similar to the modern-day pie chart to show monthly deaths and their causes during the war:

The graph shows the number of army deaths that occurred from preventable disease (in gray), those that were from the results of wounds (red), and those due to other causes (black). The circles are divided by months of the year, and months with longer wedges have more deaths. For months during the first part of the war (see the coxcomb shown to the right), the gray wedges are largest. However, for months after March 1855 (see the coxcomb to the left), the gray wedges become smaller because this is when Nightingale and the Sanitary Commission started addressing public sewage, ventilation, etc. in Turkey.

In the article, Rehmeyer brings up that others have suggested that a bar chart would have helped Nightingale make her point more “dramatically.” What do you think? Perhaps a clustered bar chart with colored bars for each cause of death (with month/year on the x-axis, and mortality rate on the y-axis)?

Regardless of which graphical method Nightingale should or shouldn't have used to prove her point, I think it’s worthwhile to note the power of statistical analysis in promoting and persuading reform. After all, her analysis—and especially her graphics—led to widespread public sanitation and medical reform, which saved countless lives.

It's all too easy to make mistakes involving statistics. Powerful statistical software can remove a lot of the difficulty surrounding statistical calculation, reducing the risk of mathematical errors—but  correctly interpreting the results of an analysis can be even more challenging. 

No one knows that better than Minitab's technical trainers. All of our trainers are seasoned statisticians with years of quality improvement experience. They spend most of the year traveling around the country (and around the world) to help people learn to make the best use of Minitab software for analyzing data and improving quality.  

A few years ago, Minitab trainers compiled a list of common statistical mistakes—the ones they encountered over and over again. Below are a few of their most commonly observed mistakes that involve drawing an incorrect conclusion from the results of analysis.  

When comparing multiple means, statistical practitioners are sometimes advised to compare the results from confidence intervals and determine whether the intervals overlap. When 95% confidence intervals for the means of two independent populations don’t overlap, there will indeed be a statistically significant difference between the means (at the 0.05 level of significance). However, the opposite is not necessarily true. CI’s may overlap, yet there may be a statistically significant difference between the means.

Take this example:

 
Two 95% confidence intervals that overlap may be significantly different at the 95% confidence level.

What is the significance of the t-test P-value? The P-value in this case is less than 0.05 (0.049 < 0.05), telling us that there is a statistical difference between the means, (yet the CI's overlap considerably).  

With statistics, we can analyze a small sample to make inferences about the entire population. But there are a few situations where you should avoid making inferences about a population that the sample does not represent:

• In capability analysis, data from a single day is sometimes inappropriately used to estimate the capability of the entire manufacturing process.
• In acceptance sampling, samples from one section of the lot are selected for the entire analysis.
• A common and severe case occurs in a reliability analysis when only the units that failed are included in an analysis and the population is all units produced.

To avoid these situations, define the population before sampling and take a sample that truly represents the population.

It’s sometimes overused, but “correlation does not imply causation” is a good reminder when you’re dealing with statistics. Correlation between two variables does not mean that one variable causes a change in the other, especially if correlation statistics are the only statistics you are using in your data analysis.

For example, data analysis has shown a strong positive correlation between shirt size and shoe size. As shirt size goes up, so does shoe size. Does this mean that wearing big shirts causes you to wear bigger shoes? Of course not! There could be other “hidden” factors at work here, such as height. (Tall people tend to wear bigger clothes and shoes.)

Take a look at this scatterplot that shows that HIV antibody false negative rates are correlated with patient age:

 
Does this show that the HIV antibody test does not work as well on older patients? Well, maybe …

But you can’t stop there and assume that just because patients are older, age is the factor that is causing them to receive a false negative test result (a false negative is when a patient tests negative on the test, but is confirmed to have the disease).

Let’s dig a little deeper. Below you see that patient age and days elapsed between at-risk exposure and test are correlated:

 
Older patients got tested faster … before the HIV antibodies were able to fully develop and show a positive test result.

Keep the idea that “correlation does not imply causation” in your mind when reading some of the many studies publicized in the media. Intentionally or not, the media frequently imply that a study has revealed some cause-and-effect relationship, even when the study's authors detail precisely the limitations of their research.

It's important to remember that using statistics, we can find a statistically significant difference that has no discernible effect in the "real world." In other words, just because a difference exists doesn't make the difference important. And you can waste a lot of time and money trying to "correct" a statistically significant difference that doesn't matter.  

Let's say you love Tastee-O's cereal. The factory that makes them weighs every cereal box at the end of the filling line using an automated measuring system. Say that 18,000 boxes are filled per shift, with a target fill weight of 360 grams and a standard deviation of 2.5 grams.  

Using statistics, the factory can detect a shift of 0.06 grams in the mean fill weight 90% of the time. But just because that 0.06 gram shift is statistically significant doesn't mean it's practically significant. A 0.06 gram difference probably amounts to two or three Tastee-O’s—not enough to make you, the customer, notice or care. 

In most hypothesis tests, we know that the null hypothesis is not exactly true. In this case, we don’t expect the mean fill weight to be precisely 360 grams -- we are just trying to see if there is a meaningful difference. Instead of a hypothesis test, the cereal maker could use a confidence interval to see how large the difference might be and decide if action is needed.

In a hypothesis test, you pose a null hypothesis (H0) and an alternative hypothesis (H1). Then you collect data, analyze it, and use statistics to assess whether or not the data support the alternative hypothesis. A p-value above 0.05 indicates “there is not enough evidence to conclude H1 at the .05 significance/alpha level”.

In other words, even if we do not have enough evidence in favor of the alternative hypothesis, the null hypothesis may or may not be true. 

For example, we could flip a fair coin 3 times and test:

H0: Proportion of Heads = 0.40 

H1: Proportion of Heads ≠ 0.40

In this case, we are guaranteed to get a p-value higher than 0.05. Therefore we cannot conclude H1. But not being able to conclude H1 doesn't prove that H0 is correct or true! This is why we say we "fail to reject" the null hypothesis, rather than we "accept" the null hypothesis.  

One final mistake we’ll cover here is not knowing when to seek the advice of a statistical expert. Sometimes, employees are placed in statistical training programs with the expectation that they will come out immediately as experienced statisticians. While this training is excellent for basic statistical projects, it’s usually not enough to handle more advanced issues that may come about. After all, most skilled statisticians have had 4-8 years of education in statistics and at least 10 years of real-world experience!

If you’re in need of some help, you can hire a Minitab statistician. Learn more about Minitab’s Mentoring service by visiting http://www.minitab.com/training/. 

Eston Martz and Michelle Paret contributed to the content of this post.

Tell us in the comments below: Have you ever jumped to the wrong conclusion after looking at statistics? 

Minitab Statistical Software offers many features that can save you time and effort when you’re learning statistics or analyzing data. However, when we demonstrate many of these short cuts, tools, and capabilities at shows and events, we find that even some longtime users aren’t aware of them.

I asked members of our sales team and technical support staff to list some of Minitab’s most helpful, yet frequently overlooked features. How many do you use—or want to start using?

Frequently, you’ll need to modify or re-run some part of an analysis you conducted. You can easily return to your last dialog box by pressing CTRL+E.  

What if you need more than 1 version of a graph?  Maybe you're presenting your results to two different audiences, and you'd like to highlight different factors for each. Use Editor > Duplicate Graph to create an identical copies of the original graph, which you can then tailor to suit each of your audiences.

It’s also easy to create new graphs using different variables while retaining all of your graph edits. With a graph or control chart active, choose Editor > Make Similar Graph to make a graph that retains all properties of your original graph but uses different columns.

To customize menus and toolbars, choose Tools > Customize. You can add, delete, move, or edit menus and toolbars; add buttons to Minitab that you can simply click on to run macros; and set keystrokes for commands.

You can specify default settings using Tools > Options. Then store all your personalized settings and customizations in a profile (using Tools > Manage Profiles) that you can use whenever you choose and share with colleagues.

Need to change the format of a column? For example, do you need to convert a text column to numeric format for your analysis?  Just choose Data > Change Data Type and select the appropriate option.

Have you ever created a graph and wished you could switch the order of the results? For instance, you might want to change “High, Medium, Low” to “Low, Medium, High”.  To display your results in a specific order, right-click on the column used to generate the output and choose Column > Value Order. This lets you set the value order for a text column using an order you define. The value order lets you control the order of groups on bar charts and other graphs, as well as tables and other Session Window output.

If you’ve never clicked on Minitab’s Help menu, you’re missing a tremendous collection of resources. Of course you’ll find guidance about how to use Minitab software there, including step-by-step tutorials. In addition, you’ll also find:

• Minitab’s Statistical Glossary.  This comprehensive, illustrated glossary covers all areas of Minitab statistics. Each definition contains practical, easy-to-understand information.
• StatGuide™. You’ve run an analysis, but what does it mean? StatGuide explains how to interpret Minitab results, using preselected examples to explain your output.
• A list of Methods and Formulas. 
• Links to helpful Internet resources, including our extensive Answers Knowledgebase.

And if you don’t find the answers you need, you can contact Minitab’s free Technical Support for assistance from highly-skilled specialists with expertise in both computing and statistics.

Do you have any favorite "hidden helpers" in Minitab? 

Minitab graphs are powerful tools for investigating your process further and removing any doubt about the steps you should take to improve it. With that in mind, you’ll want to know every feature about Minitab graphs that can help you share and communicate your results effectively. While many ways to modify your graph are on the Editor menu, some of the best features become available when you right-click your graph.

Here are the five coolest things you can do when you right-click a graph in Minitab Statistical Software.

Once your graph is ready for your report or presentation, you’ll want to put the graph in your document. Minitab makes this easy because you can right-click your graph and select either Send Graph to Microsoft Word or Send Graph to Microsoft PowerPoint. With that, you’re all set to go.

When you use the Minitab menu to transfer your graph to a presentation document, Minitab automatically selects the format that provides the clearest graph. In the case of PowerPoint, Minitab also makes sure that the graph is automatically fit to fill the receiving slide.

Getting your graph into a report is an important step, but you also want to be ready to explain your results. That’s where Minitab’s StatGuide™ comes into play. Right-click your graph, and the last menu item is always going to be StatGuide. Select StatGuide and you’ll be taken directly to a page about the graph that you’re examining. Minitab saves you the time you would have spend looking for information about the output so that you have more time to get things done.

Graphs are excellent tools for exploring and communicating, but that doesn’t mean that you never want to see the exact numbers. Getting the numbers from a graph is as easy as selecting an individual component and choosing Copy Text.

For example, you have a boxplot and would like to see the exact statistics for the graph. The tooltip for the boxplot includes the mean, quartiles, minimum, maximum, interquartile range, and sample size. Select the box with a right-click and Copy Text is active in the context menu. You can even paste a text box directly onto the graph with the information from the tooltip!

If you have a lot of graphs open, your Minitab window can sometimes get a little full. On those occasions, the right-click menu makes it easy for you to compare what you see on your graph with what’s in your data. For example, say that you’re looking at a residuals vs. fits plot and you brush the rows with the largest fits. Right-click the graph and choose Switch to, and you can quickly match up the brushed rows with the data on your graph. Here, you can easily see that while the catalyst changes between rows 4 and 8, the settings for time and temperature are the same.

Sometimes you have to produce a lot of output in Minitab to understand your data. When you get a mix of lots of statistics and graphs, you’ll want to be able to easily find the Coefficients table for a particular residual plot or the p-value for a t-test shown on a specific individual value plot. Right-clicking a graph can save you again. When you right-click and select Go to Session Line, you’re taken to the portion of the session window where the graph was made. Any tables or statistics that Minitab produced at the same time as the graph are right above that point in the session window!

Graphs are an important tool for making sure that everyone understands the results of your data analysis. When you right-click a graph in Minitab, you’ll find a number of tools that make it easier to share and understand your results. The right-click menu is one more step on your path to fearless data analysis.

In Minitab, the Assistant menu is your interactive guide to choosing the right tool, analyzing data correctly, and interpreting the results. If you’re feeling a bit rusty with choosing and using a particular analysis, the Assistant is your friend!

Previously, I’ve written about the new linear model features in Minitab 17. In this post, I’ll work through a multiple regression analysis example and optimize the response variable to highlight the new features in the Assistant.

As part of a solar energy test, researchers measured the total heat flux. They found that heat flux can be predicted by the position of the focal points. We’ll use the new features in the Assistant to correctly position the focal points.

I’ve used this example dataset for a previous post about prediction intervals. It now includes an additional variable to highlight the Assistant’s capabilities.

In Minitab, go to Assistant > Regression, and you’ll see the interactive decision tree. You can click the diamonds for more information about how to choose and for examples of the analyses.

We have three X variables (predictors) and want to fit a regression model and to optimize the response variable. Following the tree takes us to Optimize Response at the bottom right.

Our response variable is HeatFlux and the X variables are the East, South, and North focal points. From my previous post, we’ve determined that we want to target the heat flux value of 234, but the Assistant can also maximize or minimize the response. We’ll also have the Assistant help us check for interaction effects and curvature.

Click the Optimize Response button and fill in the dialog that appears with this information:

The Assistant takes our candidate X variables and produces a regression model using stepwise regression. Let's take a look at the reports that the Assistant provides.

This Summary Report tells us that our regression model is statistically significant with a P value less than 0.001 and has an R-squared value of 96.15%. Great! The comments section indicates which variables were included in the model. In this case, the Assistant includes East, South and North, along with several polynomial terms to model curvature and several interaction terms.

The Effects Report graphically illustrates all of the interaction and main effects that are in the regression model. The lines are curved when the Assistant includes a polynomial term to fit a curve.

For example, the East*South interaction is significant, which indicates that the effect one variable has on heat flux depends on the setting of the other variable. If South is set at a low setting (31.84), increasing East reduces the heat flux. However, if south is at a high setting (40.55), increasing East increases the heat flux.

The Diagnostic Report displays the residuals versus fitted values and identifies unusual points that we should investigate. Based on the criteria for large residuals, expect roughly 5% of the observations to be flagged as having a large residual. So, the two we have are not necessarily problematic. There are also two points with unusual X values. You can click the points to see which row they are in the worksheet.

The Model Building Report shows the details about how the Assistant built the regression model, the regression equation, which variables contribute the most information, and whether the X variables are correlated with each other. North contributes the most information to this model. East is not significant but the Assistant includes it because it is part of a higher-order term.

The Assistant always watches your back. For example, when it builds your model, it uses standardized X variables because standardization removes most of the correlation between linear and higher-order terms, which reduces the chance of adding these terms unnecessarily. The final model is displayed in unstandardized (natural) units.

The Prediction and Optimization Report shows the Assistant’s solutions for obtaining our targeted value of 234. The optimal settings for the focal points are East 37.82, South 31.84, and North 16.01. For these settings, the models predicts a heat flux of 234 with a prediction interval of 216 to 252. The Assistant also provides alternate solutions for you to consider using your subject area expertise.

Finally, the Report Card prevents us from overlooking issues that could make the results unreliable. For example, we should collect a larger sample size and check the unusual residuals. Normality is not an issue for our data. And, there is a reminder that we should perform confirmation runs to validate the optimal values.

The methods used in the Assistant are based on established statistical practice and theory, referenced guidelines in the literature, and simulation studies performed by statisticians at Minitab. For details, read the technical white paper for Multiple Regression in the Assistant.

If you're just learning about regression, check out my regression tutorial!

From 1985 until 2013, the NBA Finals used a 2-3-2 format to decide the NBA Champion. That is, the first 2 games are played at one team’s arena, followed by 3 in a row at the other team’s, then the last two at the first team’s. However, this year they are changing to the 2-2-1-1-1 format, where each team plays 2 of the first four games at home, then they alternate each game for games 5-7.

So how does this change affect the series? Obviously, it doesn’t change the probability of either team winning or losing. At the end of the day there are still 4 games at one team’s home arena, and 3 at the other. You can put them in whatever order you want, you won’t change the probability either team has of winning the series.

But will the order affect the number of games the series might last? To determine this, let’s assume that the two teams that meet in the finals are equal. But because of home court advantage, let’s say whichever team is playing at home has a 60% chance of winning. We can then calculate the probability that the series goes a certain number of games.

For example, for a team to win in 4 games, they have to win 4 consecutive games. In both formats, they’ll have to win 2 at home and 2 on the road. So the probability is .6*.6*.4*.4 = 5.76%.

I used Minitab Statistical Software to calculate similar probabilities for all 70 different ways a 7-game series can end. Then I added the probabilities together by the number of games the series lasts and which team won the series. Team H is the team that gets to play 4 home games (H for home) and Team A is the one with only 3 home games (A for away).

Team A wins: 2-2-1-1-1

Team A wins 2-3-2

Team H wins 2-2-1-1-1

Team H wins 2-3-2

4 games

5.76%

5.76%

5.76%

5.76%

5 games

9.98%

14.98%

14.98%

9.98%

6 games

18.24%

13.25%

13.25%

18.24%

7 games

12.81%

12.81%

19.22%

19.22%

There is a small difference in games 5 and 6. In the 2-3-2 format, the team with home court advantage has a very small chance of ending the series in fewer than 6 games. The same is true for the team without home court advantage in a 2-2-1-1-1 format. If you only get to play 2 of the first 5 games at home, your chances of ending the series in fewer than 6 games is very small. But then, you get a big advantage of getting to play game 6 at home, if the series goes that far.

When you get to the finals in any sport, the number of games the series goes doesn’t really matter. But it can matter before that. In the earlier playoff rounds, winning a series in fewer games gives your team more rest and fewer games for a potential injury. And we see that in the 2-3-2 format, the team with the extra home game will have to play more games on average than in the 2-2-1-1-1 series (assuming they win the series).

When a team has the extra home game in a 7-game series, it is often a reward for performing well in the regular season. Having a 2-2-1-1-1 format in the earlier rounds of the playoffs furthers that reward by giving the team a better opportunity to end the series in fewer games. Considering this is the format that is used in the NHL, NBA, and MLB, it looks like they’re getting it right. It gives what should be the better team the best chance of having a short series, but is still fair to the underdog (a 4-3 format wouldn't be very fair).

As for the NBA Finals, the change in formats won’t really have that much of an effect. The NBA Finals will look pretty much the same as they have the previous 29 years with the 2-3-2 format. So if the NBA really wants to change things, they would have to do something more drastic.

Perhaps a 9-game series?

by Matthew Barsalou, guest blogger

Programs such as the Minitab Statistical Software make hypothesis testing easier; but no program can think for the experimenter. Anybody performing a statistical hypothesis test must understand what p values mean in regards to their statistical results as well as potential limitations of statistical hypothesis testing.

A p value of 0.05 is frequently used during statical hypothesis testing. This p value indicates that if there is no effect (or if the null hypothesis is true), you’d obtain the observed difference or more in 5% of studies due to random sampling error. However, performing multiple hypothesis tests with p > 0.05 increases the chance of a false positive.

This is well illustrated by the online comic XKCD, which depicted somebody stating that jelly beans cause acne.

Scientists investigated and found no link, so the person made the claim that it is only a certain color jelly bean that caused acne. The scientists then test 20 different colors of jelly beans with p > 0.05. Only the green jelly bean had a p value less than 0.05.

The comic ends with a newspaper reporting a link between green jelly beans and acne. The newspaper points out there is 95% confidence with only a 5% chance of coincidence. What is wrong with the conclusion?

We can determine the chance that there will be no false conclusions by using the binomial formula.

This means that we have a 35.8% chance of performing 20 hypothesis tests without getting a false positive (or, as statisticians refer to it, the family error rate) when using an alpha level of 0.05. We can also calculate the probability that we have at least one incorrect result due to random chance.

The chance that at least one result will be a false positive when performing 20 hypothesis tests using an alpha level of 0.05 is 64.2%.

So the press release in the XKCD comic may have been a bit premature.

Suppose I had 14 samples with a mean of 87.2 and I wanted to know if the mean is actually 85.2. I performed a One-Sample T-test using Minitab by going to Stat > Basic Statistics  >  1 Sample t …. And I entered the summarized data. I checked the “perform hypothesis test box” and then selected “Options…” and used the default confidence level of 95.0. This corresponds to an alpha of 0.05.

I performed the test and the resulting p value was 0.049, which is close to but still below 0.05, so I can reject my null hypothesis. If I performed the test repeatedly, as in the XLCD example, I might have failed to reject the null hypothesis, because the 5% probability adds up with additional tests.

There are alternatives to statistical hypothesis testing; for example, Bayesian inference could be used in place of hypothesis testing with p values. But alternative methods have their own weaknesses, and they may be difficult for non-statisticians to use.

Instead of avoiding the use of hypothesis testing, we should account for its limitations. For example, by realizing that each repeat of the test increases the chance of a false positive, as illustrated by XKCD's jelly bean example.

We can’t simply retest over and over using  the same p value and then conclude that we have results with statistical significance. For situations such as in the XKCD example, Simons, Nelson and Simonsohn recommend disclosing the total number of test that were performed. Had we known that 20 test had been performed with p > 0.05 we could realize that we may not need to avoid green jellybeans after all.

xkcd.com comic from http://xkcd.com/882/ used under Creative Commons Attribution- NonCommercial 2.5 License. http://xkcd.com/license.html

Remember "The Little Engine That Could," the children's story about self-confidence in the face of huge challenges? In it, a train engine keeps telling itself "I think I can" while carrying a very heavy load up a big mountain. Next thing you know, the little engine has done it...but until that moment, the outcome was uncertain.

It's a wonderful story for teaching kids about self-confidence. But from a quality and customer service viewpoint, it's a horror story: if your business depends on taking the load up the hill, you want to know you can do it.

That's where capability analysis comes in. 

When customers ask if you're able to meet their requirements, process capability analysis lets you reply, "I know we can."

You want to determine if your part-making process can meet a customer's specification limits—in other words, can you produce good parts?  Statistically speaking, we assess the capability to make good parts by comparing the width of the variation in your process with the width of the specification limits.

The first step in capability analysis is to make sure your process is in statistical control, or producing consistently. If it's not, any estimates of process capability you make won't be reliable.

The results of a capability analysis usually include capability histograms and capability plots that help you visually assess the distribution of your data and verify that the process is in control.

It also includes capability indices, which are ratios of the specification tolerance to the natural process variation. Once you understand them, capability indices are a simple way of assessing process capability. Because they reduce process information to a single number, you can also use capability indices to compare the capability of one process with another.

This video offers a quick demonstration of a simple capability analysis:

You need to select the right capability analysis for your data based on its distribution. Depending on the nature and the distribution of your process data, you can perform capability analysis for:

• normal or nonnormal probability models (for measurement data)
• normal data that might have a strong source of between-subgroup variation
• binomial or Poisson probability models (for attributes or count data)

Minitab statistical software will help you identify the distribution that fits your data, or transform your data to follow normal distribution, before a capability analysis.

Capability analysis using a normal probability model provides a more complete set of statistics, but it assumes that the data follow an approximately normal distribution, and come from a stable process.

If you apply normal capability analysis to badly skewed data, you may drastically over- or underestimate the defects a process will produce. In this case, it's better to select a probability model based on a nonnormal distribution that best fits your data.

Alternatively, you might transform the data to better approximate the normal distribution. Minitab can transform your data using the Johnson transformation or Box-Cox power transformation.

The important thing to keep in mind is that in both normal and nonnormal capability analysis, the validity of the results depends on the validity of the assumed distribution.

Typically, data for a capability analysis consists of groups of samples, produced over a short period, that are representative of the output from the process. Collecting small subgroups of samples under the same conditions, and then analyzing the variation within these subgroups, lets you estimate natural variation in the process. You can also use individual item data to assess capability, as long as it's been collected over a long enough time to account for different sources of variation.

Guidelines typically recommend getting at least 100 total data points—such as 25 subgroups of size 4—to obtain reasonably precise capability estimates.

Process data also may have random variation between subgroups. If you think strong between-subgroup variation exists in your process, use Minitab's Capability Analysis (Between/Within) or Capability Sixpack (Between/Within) options, which calculate both within- and between-subgroup standard deviations, then pool them to calculate the total standard deviation. Accounting for both sources of subgroup variation can give you a more complete estimate of the your process' potential capability.

If you have attribute (count) data, you can perform capability analyses based on the binomial and Poisson probability models. For example, with Capability Analysis (Binomial) you can compare products against a standard and classify them as defective or not. Capability Analysis (Poisson) lets you classify products based on the number of defects.

The full range of capability tools in Minitab are found in the Stat > Quality Tools > Capability Analysis menu, including:

• Normal and Non-normal Capability Analysis
• Between/Within Capability Analysis
• Normal and Nonnormal Capability Analysis with Multiple Variables
• Binomial Capability Analysis
• Poisson Capability Analysis

You should also check out the Capability SixpackTM for Normal, Nonnormal, or Between-Within capability analyses, which combines the following charts into a single display, with a subset of the capability statistics:

• Chart to verify that the process is in control.
• Capability histogram and probability plot to verify the data follow the specified distribution.
• Capability plot that displays process variability compared to the specifications.

If capability analysis seems complicated, there's no denying that it can be. However, the Assistant in Minitab Statistical Software can take a lot of the labor and uncertainty out of doing capability analysis, especially if its a method you're new to. I'll cover how to use the Assistant for capability analysis in detail in my next post.

A recent study has indicated that female-named hurricanes kill more people than male hurricanes. Of course, the title of that article (and other articles like it) is a bit misleading. The study found a significant interaction between the damage caused by the storm and the perceived masculinity or femininity of the hurricane names. So don’t be confused by stories that suggest all female-named hurricanes are deadlier than male-named hurricanes. The study actually found no effect of masculinity/femininity for less severe storms. It was the more severe storms where the gender of the name had a significant relationship with the number of deaths.

The study looked at every hurricane since 1950, with the exception of Katrina and Audrey (those two are outliers that would skew the results). Many critics of the study believe that it is biased, since almost all of the 38 hurricanes before 1979 had female names (there were two male names in the early 50s). It’s possible that our ability to forecast hurricanes has vastly improved since the 50s and 60s. So, these critics say, the difference is simply because more people died in hurricanes back when they all had a female name.

Let’s perform a data analysis to see if that is true. We will use pre- and post-1979 to distinguish between the predominantly female-name hurricane era and the era of mixed hurricane names. I’ll use the exact same data set that was used in the study, which you can get here.

For the 92 hurricanes in the study, the number of deaths and the normalized damage was recorded. The study showed that these two variables are highly correlated, so it’s important to consider both factors. If we find there were more deaths in hurricanes before 1979, we need to make sure the reason isn’t simply because those hurricanes caused more damage (implying they were bigger storms).

We can start by using a scatterplot to plot the two variables against each other, using whether the hurricane came before or after 1979 as a grouping variable. Hurricanes that occurred during 1979 were put in the After group.

We see that the two deadliest hurricanes (Camille and Diane) both occurred before 1979. If you look below them, you’ll see that many hurricanes in both eras have caused the same amount of damage, yet resulted in far fewer deaths.

Meanwhile, the two most damaging hurricanes (Sandy and Andrew) both occurred after 1979. These hurricanes caused more than three times the damage of Camille and Diane, yet resulted in fewer deaths. This gives some credibility to the idea that our improvement in being able to predict hurricanes has resulted in fewer deaths. However, Hurricane Donna supports the opposite idea: five post-1979 hurricanes resulted in more deaths than Donna, despite causing significantly less damage. It’s hard to draw conclusions from the scatterplot.

Of course, the hurricanes labeled in the plot above are pretty rare. Most of the 92 hurricanes had normalized damage less than $30 billion and fewer than 100 deaths. The descriptive statistics below show just how much of an impact those big storms can have on an analysis.

If we look at the mean, everything becomes clear! On average, hurricanes before 1979 had 11 more deaths despite causing half a billion fewer dollars in damages. But when we look at the median, which isn’t sensitive to extreme data values, the values are almost the same. 

Part of the problem is that so many smaller storms are included. The study already concluded that the name doesn’t matter for smaller storms. So let’s just focus on the big storms. The median normalized damage for all 92 storms is $1.65 billion. I took only the storms that have caused at least that much damage (there were 47 of them) and looked at the descriptive statistics again.

Once again, the mean and median paint different pictures. The mean shows that a much higher number of deaths occurred in big storms before 1979, even though those storms caused the same amount of damage. However, this is because hurricanes Camille, Diane, and Agnes are heavily influencing the mean for deaths before 1979, pulling it up much higher than the After-1979 group. And hurricanes Sandy and Andrew  influence the mean for normalized damage after 1979, pulling it up to equal the damage before 1979.

With data this skewed, the medians are a more accurate representation of the middle of the data. The median for deaths shows that there were slightly more deaths in big storms prior to 1979. However, those storms also caused more damage, implying that could be the reason for the larger number of deaths.

And even if we ignore the fact that the hurricanes before 1979 caused more damage, a Mann-Whitney test (which compares 2 medians, as opposed to a 2-sample t test which compares 2 means) shows that the difference in deaths is not statistically significant.

The p-value is 0.1393, which is greater than 0.05. There isn’t enough evidence to conclude that hurricanes caused more deaths before 1979.

The lack of conclusive evidence from our data analysis certainly makes the idea that hurricanes with female names cause deaths plausible. But there are other issues to consider. For example, the gender of the hurricane name was not treated as a binary variable, which would group each hurricane as either male or female. Instead, nine independent coders rated the masculinity vs. femininity of historical hurricane names on two items (1 = very masculine, 11 = very feminine, and 1 = very man-like, 11 = very woman-like), which were averaged to compute a masculinity-femininity index (MFI).

Do these 9 coders represent how most Americans would rate the femininity of names? Would you rate Barbara as more feminine than Carol or Betsy? The coders did, giving Barbara a 9.8 while Carol and Betsy were 8.1 and 8.3 respectively. And the MFI is important, since it was found to be the gender variable that had a significant interaction with normalized damage. When gender name was treated as a binary variable, there was no interaction.

But masculinity-femininity index aside, the study did have some very interesting findings. I’m sure additional research will be done in the years to come to see if the findings hold true. Let's hope that then we’ll be able to know for sure whether people underestimate female-named hurricanes or not.

Until then, if a hurricane is bearing down on your neighborhood, I would make sure to board up the windows and buy out the supermarket's bread and milk, regardless of the storm's name.

You know what really gets on my nerves? A lot of things.

That slow, slinky way that cats walk by. Grrrr.

The rude, abrupt arrival of delivery persons in their obnoxiously loud trucks. (Why do they always pull up just as I’m settling down for a nap?) Grrrr.

Total strangers who reach down and poke me with fat, clumsy fingers that reek of antibacterial soap. Grrrr.

And this one always gets my dander up: Me and the human are out on a walk when some passerby  stops and points at me.

“What a cutie. How old is she?”

"What insolence!" I'll yap back. "I’m a he! Andhow old are YOU!!?"

Then I’m told to shut up.

“He’s 7.”

“Oh, so that means he’s...7*7 = 49 years old in dog years.” 

I have a bone to pick with oversimplified estimation models like this. Not every relationship can be neatly described by a straight line, you know.

I know, I know, it's not the humans’ fault. They just haven’t been properly trained. So a lot of them think they can plot a few data points on an X-Y scale and plunk a strand of raw spaghetti over it to predict whatever they darn well please. That’s something a Great Dane would do. Grrrr.

We Yorkies know that relationships can be much more complicated.

Look, I was completely potty trained at the age of 2 months. Human children, not the brightest, are lucky to get it by the time they're 3 years old. At 3 months, I could consistently comply with verbal requests to sit, stay, and lay down. Human children? Good luck with that!

In early life, there are at least 15 dog years to every human year! Maybe more.

And what about later life? Did you hear about the Australian Cattle Dog named Bluey who lived to be almost 30 years old? You mean to tell me he was over 200 years old in dog years? Grrrr.

My point? Even if a 1-to-7 relationship between human years and dog years generally holds true, you have to be doggone careful about extrapolating across all X-Y values.You need to carefully define the range of values for which your linear model holds.

Another option is to break away from the straight-jacket. Consider using a quadratic or cubic function to model data that don't toe the line:

Being a fearless Yorkie, I’m not even afraid to consider using a more advanced nonlinear function, if it can provide a significantly better fit than a linear regression model.

Like a sigmoidally shaped function:

a concave function:

or a function with local minima and maxima:

I could yap on and on about this, but then I'll be told to shut up. You can find these and other nonlinear functions in Minitab's catalog of functions for nonlinear regression (Stat > Regression > Nonlinear Regression > Use Catalog).

There's another thing about that Human Years/Dog Years graph that really makes me froth at the mouth. It treats all dogs as a single population. And we all know what happens when you make an inference based on single population when, in fact, your data are actually from distinct populations.

You get a one-size-fits-none estimate. Grrrrr.

The obvious trend in this bar chart makes my point:

How could humans use the same model to estimate the "dog age" of an Irish Wolfhound (average lifespan ≈ 6 years) and a miniature poodle (average lifespan ≈ 15 years)?!

Do me a favor. Next time, instead of overestimating my age, at least use a model with a grouping variable that accounts for these distinct populations, would you?

This model isn't perfect either. I, for one, certainly don't appreciate being lumped together with dogs that weigh twice as much as I do. 

And I hate to sound catty, but compared to some other breeds, I think I look much younger than my age.

Toolbars are there to make your life easier, but if you don’t take the time to hover over each button and wait for a description, it’s pretty easy to never know that there’s a faster way to do something.

The toolbars in Minitab Statistical Software include some pretty nifty shortcuts. Here are my favorite 6:

1. StatGuide

As soon as you have results in Minitab, the StatGuide button becomes active on your toolbar. Click the button, and the StatGuide opens directly to guidance for the analysis that you’re looking at. Minitab saves you the time you would have spent looking for information about your results so that you have more time to get things done.

2. Edit Last Dialog

To repeat an analysis, either because you want to run it on a different column or because you want to change a setting, all you have to do is click a button. Even better, most of Minitab’s analyses will remember what you entered the last time the dialog box was open. Make the small adjustments you need to make, and you’re ready to perform your new analysis.

3. Show Session Folder

When you’ve run several analyses in Minitab, it can be nice to have a quick way to find the results of a particular analysis. Minitab’s project manager is the best way to find the results of an analysis quickly, and that’s why it’s so nice that it’s accessible from the toolbar. Click the button, and you get a list of all of the analyses and graphs in your Minitab project.

4. Current Data Window

If you have a lot of worksheets open, you might want to be able to see both your worksheet and your results at the same time. When you click the button, the current worksheet comes to the front, without maximizing to hide your results. Click it again, and the next worksheet comes to the front. Click it again, and the next worksheet comes to the front. You can quickly cycle through the worksheets to find the one that you want, while still being able to see the results from your analysis.

5. Assign Formula To Column

The Minitab calculator’s a nice tool, but with the toolbar, you can use it even faster. The best part of all is that when you use the toolbar, you specify which column will have the formula without having to tell the calculator. Plus, when you’re in a complicated series of formulas, the column where the formula goes is not in the list of columns to select, so you can never get a recursive formula error.

6. Show Info

Especially after you open or copy data from Excel, it can be helpful to get a quick snapshot of the columns in your worksheet. When you click the Info button, the Project Manager shows the column names, the lengths, the number of missing values, and the format of the columns. You can investigate why a column that contains numeric data is formatted as text. If any of the columns are the wrong length because of missing values at the end, you know right where to look. You won't have to spend your time scrolling around the worksheet looking for things that are amiss, so that you can get to your analysis faster.

The only thing better than doing fearless data analysis is doing fearless data analysis even faster. Minitab’s toolbars come ready with shortcuts that help you analyze your data faster, from generating your results to interpreting them. Of course, the toolbars that everyone uses can’t be perfect for everyone. If you’re feeling emboldened, check out how to customize the existing toolbars or even to create your own toolbars!

In regression analysis, you'd like your regression model to have significant variables and to produce a high R-squared value. This low P value / high R2 combination indicates that changes in the predictors are related to changes in the response variable and that your model explains a lot of the response variability.

This combination seems to go together naturally. But what if your regression model has significant variables but explains little of the variability? It has low P values and a low R-squared.

At first glance, this combination doesn’t make sense. Are the significant predictors still meaningful? Let’s look into this!

It’s difficult to understand this situation using numbers alone. Research shows that graphs are essential to correctly interpret regression analysis results. Comprehension is easier when you can see what is happening!

With this in mind, I'll use fitted line plots. However, a 2D fitted line plot can only display the results from simple regression, which has one predictor variable and the response. The concepts hold true for multiple linear regression, but I can’t graph the higher dimensions that are required.

These fitted line plots display two regression models that have nearly identical regression equations, but the top model has a low R-squared value while the other one is high. I’ve kept the graph scales constant for easier comparison.

The two models are nearly identical in several ways:

• Regression equations: Output  =  44 +  2 * Input
• Input is significant with P < 0.001 for both models

You can see that the upward slope of both regression lines is about 2, and they accurately follow the trend that is present in both datasets.

The interpretation of the P value and coefficient for Input doesn’t change. If you move right on either line by increasing Input by one unit, there is an average two-unit increase in Output. For both models, the significant P value indicates that you can reject the null hypothesis that the coefficient equals zero (no effect).

Further, if you enter the same value for Input into both equations, you’ll calculate nearly equivalent predicted values for Output. For instance, an Input of 10 yields a predicted Output of 66.2 for one model and 64.8 for the other model.

I bet the main difference is the first thing you noticed about these fitted line plots: The variability of the data around the two regression lines is drastically different. R2 and S (standard error of the regression) numerically describe this variability.

The low R-squared graph shows that even noisy, high-variability data can have a significant trend. The trend indicates that the predictor variable still provides information about the response even though data points fall further from the regression line. Keep this graph in mind when you try to reconcile significant variables with a low R-squared value!

As we saw, the two regression equations produce nearly identical predictions. However, the differing levels of variability affect the precision of these predictions.

To assess the precision, we’ll look at prediction intervals. A prediction interval is a range that is likely to contain the response value of a single new observation given specified settings of the predictors in your model. Narrower intervals indicate more precise predictions. Below are the fitted values and prediction intervals for an Input of 10.

The model with the high variability data produces a prediction interval that extends from about -500 to 630, over 1100 units! Meanwhile, the low variability model has a prediction interval from -30 to 160, about 200 units. Clearly, the predictions are much more precise from the high R-squared model, even though the fitted values are nearly the same!

The difference in precision should make sense after seeing the variability present in the actual data. When the data points are spread out further, the predictions must reflect that added uncertainty.

Let’s recap what we’ve learned:

• The coefficients estimate the trends while R-squared represents the scatter around the regression line.
• The interpretations of the significant variables are the same for both high and low R-squared models.
• Low R-squared values are problematic when you need precise predictions.

So, what’s to be done if you have significant predictors but a low R-squared value? I can hear some of you saying, "add more variables to the model!"

In some cases, it’s possible that additional predictors can increase the true explanatory power of the model. However, in other cases, the data contain an inherently higher amount of unexplainable variability. For example, many psychology studies have R-squared values less that 50% because people are fairly unpredictable.

To help determine which case applies to your regression model, read my post about avoiding the dangers of an overly complicated model.

The good news is that even when R-squared is low, low P values still indicate a real relationship between the significant predictors and the response variable.

If you're learning about regression, read my regression tutorial!

The Minitab Fan section of the Minitab blog is your chance to share with our readers! We always love to hear how you are using Minitab products for quality improvement projects, Lean Six Sigma initiatives, research and data analysis, and more. If our software has helped you, please share your Minitab story, too!

The data taken for this study is obtained from the official website of Hyderabad Traffic Police (http://www.htp.gov.in/Default.htm). Also note that the data for 2014 covers only the period until April.

Reviewing the time series plot I obtained using Minitab 17 indicates that the number of accidents steadily decreased every year from 2011-2013, but there seems to be a rise from January-April 2014.

As I was brought up in city of Hyderabad, my experience has been that the following factors influence road accidents here:

• Increasing vehicle population leading to heavy traffic during peak hours
• Drunken driving
• Speed limit violation
• Lack of properly laid roads
• Violation of traffic and safety rules
• Roads getting water logged during rainy season
• Using cell phone while driving
• Not wearing seat belts
• Unwanted hurrying/negligence of the driver
• Inattention while backing the vehicle
• Not getting clear picture of surroundings—lack of signage
• Using high beam light
• Driving without a helmet
• Speed driving on the flyovers and the Outer Ring Road
• Tripping of heavy load vehicles in the city during the day time

Following is a time series plot of the 852 accidents that took place from January-April 2014 according to the days of the week. This graph clearly indicates that the number of accidents occurring over the weekends is high.

The increase in the number of accidents over the weekend is a serious concern which requires attention since these accidents may be preventable by awareness campaigns targeted to the youth of the city.

Based on the results of the above analysis, preventive actions that I believe could be taken by the concerned authorities are:

1. Make citizens aware of the importance of strictly adhering to the traffic rules, and impose fines on those who do not abide by them.
2. Issue driving licenses only as per age limits, and only after the person clears all the tests.
3. Inspect vehicles to make sure they are road-worthy.
4. Increase the number of traffic police in areas of heavy traffic.
5. Make sure the timers installed at traffic signals function properly.
6. Analyze the major accident-prone areas scientifically to reduce the rate of occurrence.
7. Check medians, footpaths, and curvatures carefully.
8. Use paint to clearly mark humps on the roads.
9. Remove attention-seeking boards, banners, and advertisements.

Dhatry Yaso Kala
Independent Consultant and Lean Six Sigma Black Belt
Hyderabad, India
 

In an earlier post, I shared some great hidden helpers in Minitab Statistical Software that even many veteran users don't know about. Here are a few more!

Minitab’s Project Manager allows you to navigate, view, and manipulate various parts of your project. Right-clicking either the folders or their contents lets you access a variety of menus that allow you to manage Session Window output, graphs, worksheets, command language, and related project areas. You can also copy any or all analyses and graphs to Minitab’s built-in ReportPad to create reports and share your results with colleagues who may not have Minitab. You can also move ReportPad contents to a more powerful word processing program for further editing and layout. Press CTRL+I to access Minitab’s Project Manager.

To change the color of bars and symbols on a graph, double-click on any bar or symbol. Then use the Attributes tab to change the fill pattern, color, outline, and other aspects of how your graph appears.

To import data from Excel into Minitab, choose File > Open Worksheet, select Excel from Files of type and navigate to your Excel document. If you open a workbook with multiple sheets, each Excel sheet opens into a separate Minitab worksheet.

You can also click Options and Preview to specify the data to import and to ensure that it is formatted properly. Need to import a CSV or text file? You can import those and other data file formats using File > Open Worksheet.

Minitab’s Autofill capability lets you automatically repeat the value in a cell (or cells), or follow a pattern as you click and drag down a column or columns in a worksheet.

To repeat a value, select the cell, then hover your cursor on the lower right corner of the cell. When the cross appears, click and drag down the column to repeat the value in other cells.

To repeat a pattern of values rather than a single value, press CTRL as you drag, following the same convention as in Excel.

You can even use Autofill with custom lists. For example, with the worksheet active, choose Editor > Define Custom Lists to define “Mon Tues Wed Thu Fri Sat Sun” as a list, type one of the values in the first row of a column, and click and drag to fill in the list.

What features in Minitab do you find most helpful?  Let us know and we’ll share your favorites.   

There is more than just the p value in a probability plot—the overall graphical pattern also provides a great deal of useful information. Probability plots are a powerful tool to better understand your data.

In this post, I intend to present the main principles of probability plots and focus on their visual interpretation using some real data.

In probability plots, the data density distribution is transformed into a linear plot. To do this, the cumulative density function (the so-called CDF, cumulating all probabilities below a given threshold) is used (see the graph below). For a normal distribution the CDF will look like an S shape. In order to transform this S shaped curve into a line, a special Gausso-arithmetic (nonlinear) scale is needed (for the vertical Y scale).

A low p value indicates that the normality hypothesis needs to be rejected. But I want to focus specifically on analyzing graphical patterns in probability plots, based on a subjective visual examination of the data. In assessing how close the points are to a straight line, the "fat pencil" test is often used. If the points are all covered by this imaginary pencil, then the hypothesized distribution (the normal distribution in this case) is likely to be appropriate.

Probability plots may be useful to identify outliers or unusual values. The points located along the probability plot line represent “normal,” common, random variations. The points at the upper or lower extreme of the line, or which are distant from this line, represent suspected values or outliers. 

Outliers may strongly affect regression or ANOVA models since a single outlier may result in all predictor coefficients being biased. So probability plots on residual values from a statistical model are very useful for model validation and to detect some outliers that might be caused by failed tests, wrong measurements etc.

Probability plots also help up understand experimental designs. In a DOE (design of experiments) analysis, the effect plots are probability plots that represent factor or interaction effects. They may be used to identify significant effects. Effects that lie along the normal probability plot line are not significant (these effects are only caused by random variations), whereas the points that look like outliers represent real significant effects.

In the graph below, the data has been generated from an extremely asymmetrical (exponential) distribution. Clearly the points do not follow the probability plot line, with more dispersion on the longer (right-sided) tail. The data are very concentrated and close to one another at the other end (left side) of the distribution. The final result is a curve, not a line.

This has use in capability analyses: such a curvilinear pattern indicates that an asymmetrical distribution would be more appropriate (not the normal one). Cp and Cpk estimates are very sensitive to non-normality issues.

In a DOE or in a regression analysis, a plot like this this indicates that you need either to transform your data (into a normal distribution) or use another, more appropriate distribution.

In the graph below, distinct groups of points are displayed along the probability plot line. Clearly this does not represent a continuous distribution. In measurement system analysis, such a pattern often indicates that the resolution of a measuring device is insufficient (with a low number of distinct categories, for example). There are certainly small differences between the points that look totally similar in this graph, but the measurement device cannot detect and recognize such small differences, resulting in a discrete distribution.

Laser games have become very popular entertainment over the last few years. Players score points by tagging targets with an infrared device. Successfully tagging players from competing teams will increase your score, while shooting players from your own team or being targeted and shot by the opposing team will decrease your score. I collected scores from several laser game rounds to analyze the graphical patterns in the probability plots of these scores. (This is real data—my son is a fan of laser games and I collected some of his score sheets).

In graph below, the game 1 probability plot (upper left corner) has a clear outlier/suspect value (the graphs shows a “super player” in the game clearly over-performed his opponents).

Even without the outlier, note that it is not possible to draw a single “fat pencil” line across all points. We do have different groups, so three lines are needed. The three points on the left represent a group that under-performed; the points on the right group represent a group over-performed; and the group between these represents the average players. Points located along a single line represent random variation (within a normal distribution) due to “common causes.” Differences between single lines represent “special causes” (real significant differences), probably due to different fighting techniques, level of expertise, scoring ability, players that are more advanced in the learning curve etc.

Looking at all four graphs, several groups of players can clearly be distinguished (several fat pencil lines are needed in each case).

Note that the lines for the lower scores are generally steeper (less dispersion and more concentrated data), whereas the slopes of the lines for the best performing groups on the right side are more moderate, with more intra-group dispersion. This pattern indicates that the amount of dispersion is not constant according to the average value of each group (the larger the average, the larger the amount of variation: this particular behavior is named “heteroscedasticity”). This is probably because inexperienced players get shot easily and cannot express their real, full potential, whereas differences in playing styles and techniques have a more important role for more experienced players, so we see larger differences between them in terms of scores. If you've ever seen or played a laser-tag game, you can imagine beginners advancing in a huge dark space being targeted by more experienced players (well positioned behind obstacles) and not knowing where to go exactly.

How would graphs like this be interpreted in a quality improvement situation?

Capability analyses: In capability analyses, such different “fat pencil” or broken lines indicate that parts from different production lines, suppliers etc., have been mixed together. The result is an overall nonnormal distribution (with biased Cp and Cpk estimates) made from different groups that actually follow several normal distributions (with different means) mixed together. One consequence of this is that, probably, no theoretical distribution (Weibull, lognormal distributions etc.) will ever fit such data. A more complex approach—maybe a non-parametric—may be needed in this case.

Designed experiments: In a DOE effects plot, an outlier will cause some effects to be overestimated (if there is a 1 coded level in the outlier row for such a factor) or underestimated (if there is a -1 coded level in the outlier row for this particular factor). The result will be different “fat pencil” lines (broken lines) again.

Reliability: In a reliability analysis, broken lines are often caused by different failure modes. Although the normal distribution is less likely to be used in this context, Weibull probability plots convey the same type of information.

Laser games are great in that they do provide a lot of data.  I wish “Super Player” good luck in his next rounds!

As you can see, simple visual examination of probability plots does yield some useful insight into the data structure. The amount of data being generated from the Internet, electronic devices, customer relations management, and other technologies is rapidly increasing. Recognizing patterns in all this data is key to better understanding our processes, our customers, our products, and our opportunities.

Laser tag photo by Johannes Gilger, used under Creative Commons 2.0 license.

The 2014 World Cup has gotten off to a high-scoring start. Through the first week of the tournament, an average of 2.9 goals have been scored per game, the highest since 1970. And if that average climbs to over 3 goals per game, this’ll be the highest scoring World Cup since 1958!

So is this year’s World Cup actually bucking a trend of the low scoring tournaments that came before it, or can we simply attribute it to random variation? Let’s use a data analysis to find out!

I went to FIFA’s website and collected the goals per game in every world cup. Now that we have the data, let’s start by looking at a time series plot.

The time series plot shows that this year isn’t too much of an outlier. Sure, it’s the most goals per game since 1970, but it’s hardly blowing the other years out of the water. The 1982 World Cup had 2.8 goals per game, and both 1994 and 1998 had 2.7 goals per game. When compared to those years, 2.9 goals per game in this year’s World Cup doesn’t sound all that incredible.

Now, if the average returned to pre-1962 levels, that would be a lot of goals. The first six World Cups were significantly higher scoring than the game today. Because there was a clear shift in the mean before 1962, I’ll to make another time series plot looking at only the more recent World Cups that have had a similar amount of goals scored per game.

Now that we’ve removed the first six World Cups, we’re able to look at the plot on a smaller scale. And it does appear that goals per game was decreasing throughout the years before 2014. We can use Minitab’s Trend Analysis to see if this is, in fact, true.

There is a downward trend in the number of goals per game from 1962 to 2010. Instead of including 2014, I used the model to forecast what we could expect for this year using the data from the previous years. It gave us a forecast of about 2.3 goals per game, which is the same as the previous two years.

So now we’ve established that the 2.9 goals per game is bucking a downward trend. But is the difference between the forecast (2.3) and the observation (2.9) just random noise, or is there some factor in this year’s World Cup that is causing more goals to be scored?

The trend analysis above simply uses a regression analysis with the average number of goals per game as the response and the order of the observations (1, 2, 3, etc….) as the predictor. That means we’d get the same result if we did a fitted line plot using the order of the data instead of the year for the X axis.

And now we can use this regression model to give us a prediction interval for the 14th observation, which would correspond to the year 2014 in our trend analysis above.

Just like the trend analysis, we see that we would expect there to be 2.3 goals per game. And the prediction interval indicates that we can be 95% confident that the average number of goals for 2014 will be between 1.86 and 2.75 goals per game. Hey wait...2.9 goals per game is outside this interval. Could 2014 be special after all?

NOTE: I used the prediction interval instead of the confidence interval because we’re dealing with a single observation (2014) as opposed the average of multiple years. If you want more information about the difference between a confidence and prediction interval, I suggest reading this blog post.

Before we go concluding that scoring is on an upward trend in the World Cup, we need to remember the sample size. Our observation for 2014 is only for a eight days' worth of games, whereas the other data values come from the entire tournament. Through the first 4 days of this year's World Cup, the average goals per game was 3.5. For the next 4 the average was 2.4. The abundance of high-scoring games could have been simply random variation in a small sample of the first few games, and unlikely to continue.

But we won’t know for sure until we get some more data! So let’s wait two weeks and come back to see how the tournament is going. Until then, I leave you with this...

by Matthew Barsalou, guest blogger 

I told a friend about my interest in statistics, and he immediately told me a joke about broiled chicken and statistics.

The punch line involved my friend getting to eat all the chicken. Unfortunately, I forgot the rest of the joke. I can, however, assure you it was a very funny statistics-related joke.

People often make jokes when I mention my interest in statistics, and I don't think they make the jokes just because there are so many great statistics-related jokes available. There might be some good jokes about statistics, but I only know two and can only remember one.

I also don't think people make jokes about statistics because it is an inherently hilarious subject. It can be an interesting subject, but it is seldom a funny subject.

In fact, the subject can be deadly serious. In public health, statistics can be used to identify cancer clusters and to validate the effectiveness of medical tests. Failing to identify a cancer cluster or the presence of a disease or disorder in an individual could result in a medical problem going untreated.

There is also the opposite risk—falsely identifying a cancer cluster in a community or a disease in an individual. This would mean resources are wasted on healthy people, as well as the negative consequences which could result when a healthy person is given an incorrect terminal diagnosis. Being falsely diagnosed with terminal cancer is more than just statistics; it is a life-changing, personal tragedy for the person who was misdiagnosed.

For those of us using statistics in manufacturing, the consequences of improper use of statistics may not be as severe as in the medical field. It is still a serious subject. An improperly performed or simply flawed study could result in a product that angers formerly loyal customers, as the Coca-Cola Company learned when they introduced New Coke in 1985. The correct use of statistics can have series consequences for the safety of consumers and a company's financial well-being if a statistical study fails to identify a serious hazard in a product. There can also be financial consequences if a study incorrectly identifies a safety hazard where none exists.

Statistics are used in medical testing to determine both whether or not potential new medicines work, and to determine if they have unwanted side effects. Statistics are also used to determine if the benefits of some medicines outweigh the risks of using them. Here, an incorrect interpretation of statistical data could result in harming people with medicine that that was intended to help them.

The consequences of making a mistake when using statistics in business are not always severe; however, they could be. An improperly analyzed Student’s t-test may result in an implementing an expensive improvement that actually does not change anything about the product. Or it could result in the product unknowingly becoming less safe than it was before the improvement was implemented.

We may not even realize when the consequences of a statistical mistake could be severe.

For those of us who use statistics, but are not trained statisticians, fortunately there are resources available to help us in correctly selecting and applying statistical methods. The National Institute of Standards and Technology collaborated with the semiconductor industry's SEMATECH to produce a free online statistics handbook. Statistical practitioners can also attend training by universities, professional societies and industry. Practitioners can also attend training offered by Minitab.

To make up for the seriousness of this subject—as well as my inability to remember the statistics joke I mentioned at the start—I'll finish with this classic statistics joke form the Internet Joke Database:

Three statisticians went duck hunting. A duck flew out and the first statistician took a shot, the shot went a foot too high. The second statistician took his shot and the shot went a foot too low. The third statistician said, "We got it!"

Do you have a favorite statistics joke—or a cautionary tale? 

Control charts are some of the most useful tools in statistical science. They track process statistics over time and detect when the mean or standard deviation change from what they have been. The signals that control charts send about special causes can help you zero in on the fastest ways to improve any process, whether you’re making tires, turbines, or trying to improve patient care.

I’ve mentioned before that I’m a baseball fan. For the past several years, I’ve been noticing articles about the Year of the Pitcher in Major League Baseball (2010, 2011, 2012, 2013, 2014). That repetition suggests a shift to me, and I thought “What a great way to illustrate some neat things that you can do with control charts!”

Here are a few things to remember about control charts that you can illustrate with Major League Baseball data from 1969 to 2013, courtesy of numbers from www.baseball-reference.com.

We'll start in 1969 because that’s when new rules decreased the height of the pitcher’s mound in Major League Baseball parks. However, there have been some other notable changes in the game over the years that mean that we have to be sensible about the data that we plot. For example, if we make an I-MR chart of hits, we see some special causes right away:

Four points are out of control on the MR chart because of strike-shortened seasons in 1981 and 1994. One technique when you know the reason for an out-of-control point is to exclude those samples from calculating the control limits. That way, the control limits represent expected process variation. But in statistics we like to use as much of our data as possible. If you want to keep the data from those years, an alternative to throwing them out would be to plot a different variable. I used Minitab’s Calculator to create a column that contains the number of hits per at bat.  

The control chart above shows the number of hits per at bat. You still see some out-of-control points on the chart, but they no longer correspond to the strike-shortened seasons. The first out-of-control point is 1972. Not coincidentally, the American League instituted the designated hitter in 1973. A corresponding increase in hits per at bat makes sense. The next out-of-control signal comes in 1994. The most popular explanation, given that 4 of the next 7 points are out of control, is that this marks the beginning of the steroid era in baseball. The steroid theory explains that, beginning in 1994, increased use of performance-enhancing drugs reached a tipping point so that the effects of drugs were statistically visible in the game. Another explanation is that 1993 is when baseball began playing games in the thin air of Colorado, where Mile High Stadium was a hitter-friendly precursor to Coors Field.

In cases like this, you have to decide whether it’s fair to compare all of this variation on one chart or not. If you know that there has been a change in the rules, then you would expect to see corresponding out-of-control points. In fact, we might not be getting enough out-of-control points to show the changes precisely.

The same logic applies to any process: typically, you want to calculate control limits from a stable baseline. For example, if you calculate the control limits using the years 1973-1993, then 3 of the 4 years without the designated hitter are out-of-control and 6 of the 15 years 1994 to 2007 are unusual. The out-of-control points show when the process was different from the baseline years 1973-1993:

If you calculate the control limits using the years 1994-2013, then the MR chart shows precipitous changes in 1973 and 1993:

Of course, when you have different things to compare, you might want to look for points that are unusual relative to the process that they should fit. For example, you would want different control limits before and after you improve a process. Minitab’s Assistant Menu makes this easy with Before/After control charts. With the baseball statistics, a before/after control chart lets us look for points that are unusual within an era. Let’s set the dividing line at 1993 and just use the post-designated hitter years:

In the first era, there are no out-of-control points. In the second era, the years 2010-2013 are unusual, marking the return of pitching domination so many people notice. The Assistant Menu also performs a statistical test to verify that the average batting average is statistically greater in the second era than in the first.

Control charts are a powerful tool for understanding your processes. Minitab makes control charting easy, whether you want to compare different eras in baseball or different phases in your process. And the Assistant Menu makes comparisons even easier by providing all of the information you need in a single report, ready for you to export to a presentation.

Ready for more? Check out our webcast on using control charts!

Photograph of Donald "Zack" Greinke by Keith Allison, used under Creative Commons Attribution-Share Alike 2.0 Generic license.