Performance Comparisons and Hypothesis Testing
Executives feel uneasy about making performance comparisons that may lead to an inappropriate decision. This is particularly so when the recipient of that decision receives a severe penalty. Management information reduces the feeling of inadequacy that often accompanies such executive actions. Statistical analysis, an essential element of management information, is made up of inference and hypothesis testing. Statistical inference predicts the most likely value measurements can occur. Hypothesis testing, meanwhile, determines with statistical certainty the difference between two sets of observations.
Performance comparisons and other applications of hypothesis testing begin by defining their null hypotheses. These statements claim no or null difference between any two sets of data being compared. Any two sets of data came from the same underlying normal distribution. The calculated testing criterion as compared to the critical value proves with objectivity the lack of difference. When the sample size is twelve or less, the conclusion from testing may not be robust. In that case, testing may be repeated if the cost of additional data is not prohibitive. If the criterion calculated cannot support the null hypothesis, then the alternative holds.
ANOVA. Analysis-of-variance, the most accessible statistical analysis tool is included in all Microsoft Excel. The county administrator in reference (1), wanted to confirm whether the difference between the average form completion rate of 15.2% (with a variance of 13.4%) at East Service Center and the form completion rate of 22.0% (with a variance of 10.0%) at West Service Center is significant. The Null hypothesis, in this example, assumes that these data came from the same underlying distribution and the difference in averages is merely by chance. The analysis allowed the conclusion that management of the East Center did not perform as well as the management of the West Center. Additional statistical analysis refuted that conclusion. The lower completion rate at the East Service Center was due to higher client loading.
Purpose of Statistical Analysis - To determine whether the average form completion rate at the West Service Center of 22.9% is statistically better than East Service Center’s performance of 15.2%.
Null Hypothesis – There is no statistical difference in the % form completion rates between the two Service Centers. The ANOVA calculated (between groups) p statistic is equal to or greater than the critical p of 0.05.
Alternative Hypotheses – There is a statistical difference. The calculated p of 0.00 is less than 0.05.
Conclusion – With a calculated p of 0.00, the analysis supports the alternative hypotheses. West Service Center has a better admission form completion rate than East Service Center.
CHI Squared Test. The administrators of Moon Lake and Midwest counties, in reference 2, wanted to know whether counties are similar such that their respective operating ratios can be shared. They chose the count of residents in each demographic category as sound measurement. The Null Hypothesis fundamentally stated that the actual count of county residents in each category minus the corresponding expected count of residents is zero.
Purpose of Statistical Analysis - To determine whether the age distributions of residents in both counties are associated.
Null Hypothesis – The age distribution of residents among four age categories in Moon Lake County is associated with the distribution in Midwest County. The p statistic derived from calculated chi-square and degree-of-freedom is equal to or greater than the critical p of 0.05. (It is a null hypothesis in that there is no difference in the age distribution of residents.)
Alternative Hypothesis – The age distributions of residents between the two counties are not associated. The calculated p is less than the critical p. (The age distributions are different.)
Conclusion – The analysis supports the alternative hypotheses. The age distributions of residents between the two counties are not associated. The calculated p of 0.00 is less than 0.05.
Regression Analysis. In reference 3, a county administrator wanted to estimate funding needs for forecasted population growth. Within a state, funding is generally based on the number of residents a county has. This county's policies, however, require rigorous estimation procedures. Reference 4 includes hypothesis testing in multiple regression analysis.
Purpose of Statistical Analysis - to develop a regression line with the five-year average human services funding per capita as the dependent variable and the five-year average county residents its independent variable. There were eighty plus counties of varying sizes.
Null Hypothesis – the coefficients of the independent variable (1.839 in the above example) and the y-intercept are equal to zero. The regression calculated the respective p-value to be equals to or greater than the critical p of 0.05.
Alternative Hypothesis – the coefficients of the independent variable and the y-intercept are statistically different from zero. Both calculated p's were less than critical p.
Conclusion – The analysis supported the alternative hypotheses for the coefficient and the y-intercept. Their p-statistics were calculated to 0.00 and 0.04, respectively. The 5-year average funding is associated with the 5-year average number of county residents.
When making performance comparisons and other hypothesis testing applications, we suggest closely following the template statements above until one is confident in its use. Free Excel testing templates are available upon request at contact@mgmtlaboratory.com.
By Mgmtlaboratory.com Staff, 2021
References
ANOVA: An Easily Accessible Executive Analysis Tool. www.mgmtlaboratory.com, January 2018.
County Performance Comparisons: Chi-Square Testing for Similarity. www.mgmtlaboratory.com, August 2019.
SLRA (Simple Linear Regression): Multi-Purpose and Accessible Executive Analytics Tool. www.mgmtlaboratory.com, May 2018.
Evaluating Human Services Performance across Counties: Multiple Regression Analysis (MRA) Demonstration. www.mgmtlaboratory.com, July 2018.
Comments