Selecting the wrong statistical test is more common than most researchers admit. A 2026 systematic review of clinical prosthodontic publications found that statistical misapplications and reporting deficiencies persisted across all five assessed domains — including test selection, assumption testing, and result interpretation — despite decades of published guidance.1
This guide provides an interactive decision tool (above) plus a visual decision tree, quick reference table, and full explanations for every major test family. Use the tool to get your recommended test in under a minute, then read the relevant section for full reporting guidance.
Statistical Test Decision Tree
Purple = goal | Blue = decision nodes | Green = parametric | Orange = non-parametric
Quick Reference Table
| Test | When to use | Effect size | EditVerse service |
|---|---|---|---|
| Independent t-test | 2 indep. groups, continuous DV, normal | Cohen’s d | Manuscript Stats Service |
| Paired t-test | Same subjects, 2 time points, normal diff | Cohen’s d | Manuscript Stats Service |
| Mann-Whitney U | 2 indep. groups, ordinal or non-normal | Rank-biserial r | Manuscript Stats Service |
| Wilcoxon Signed-Rank | Paired, ordinal or non-normal differences | Rank-biserial r | Manuscript Stats Service |
| One-way ANOVA | 3+ indep. groups, continuous DV, normal | η² or ω² | Manuscript Stats Service |
| Kruskal-Wallis | 3+ indep. groups, ordinal or non-normal | η² | Manuscript Stats Service |
| Chi-square / Fisher’s Exact | 2 categorical variables; Fisher if expected counts <5 | Cramér’s V | Manuscript Stats Service |
| Pearson r | 2 continuous, normal, linear relationship | r with 95% CI | Manuscript Stats Service |
| Spearman ρ | Ordinal, non-normal, or monotonic relationship | ρ with 95% CI | Manuscript Stats Service |
| Linear regression | Continuous outcome, 1+ predictors | R², β with CI | Advanced Statistical Analysis |
| Logistic regression | Binary outcome, 1+ predictors | OR with 95% CI | Advanced Statistical Analysis |
| Cox PH regression | Survival / time-to-event with censoring | HR with 95% CI | Advanced Statistical Analysis |
| Poisson / Neg-binomial | Count data; neg-binomial if overdispersed | IRR with 95% CI | Advanced Statistical Analysis |
EditVerse Statistical Services
Know your test — now get it done right
The Four Decision Criteria
Every statistical test selection reduces to four questions, applied in order:23
- What is your research question? — Comparing groups, examining a relationship, or predicting an outcome?
- What type is your dependent variable? — Continuous, categorical, ordinal, or count?
- How many groups or predictors? — Two groups, three or more, one predictor, multiple predictors?
- Does your data meet the test’s assumptions? — Normality, independence, homogeneity of variance?
Why Test Selection Errors Are Common — and Consequential
Statistical errors in published research are not rare events. The 2026 review by Ben Suleiman et al. found inadequate assumption testing and inappropriate test selection among the most frequent deficiencies.1 The most commonly documented errors:
- Multiple t-tests instead of ANOVA — inflates the family-wise Type I error rate to approximately 14% with three groups
- Parametric tests on ordinal Likert data — violates the equal-interval assumption; Mann-Whitney U or Wilcoxon is required
- Skipping assumption checks entirely — especially with small samples
- Chi-square with expected cell counts below 5 — Fisher’s Exact Test is required
- Confusing correlation with regression — correlation cannot adjust for confounders
Parametric vs. Non-Parametric: When to Use Each
Parametric tests (t-test, ANOVA, Pearson r, linear regression) assume normality and roughly equal variance across groups.4 They are more statistically powerful when assumptions are met.
Non-parametric tests (Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Spearman) operate on ranks and make no distributional assumptions. They are the correct choice when data is ordinal, the sample is small and non-normal, or influential outliers are present.4
| Parametric Test | Non-Parametric Equivalent | Use Case |
|---|---|---|
| Independent t-test | Mann-Whitney U | Two independent groups, continuous DV |
| Paired t-test | Wilcoxon Signed-Rank | Two paired measurements, continuous DV |
| One-way ANOVA | Kruskal-Wallis | Three or more independent groups, continuous DV |
| Repeated-measures ANOVA | Friedman test | Three or more repeated measurements, same subjects |
| Pearson correlation | Spearman correlation | Relationship between two variables |
Checking Assumptions: Visualization Over Significance Tests
A 2024 study by Shatz (University of Cambridge) demonstrated that running Shapiro-Wilk as a binary pass/fail gate is fundamentally flawed — oversensitive with large samples, underpowered with small ones.5 Combine it with Q-Q plot inspection and treat assumption checking as a continuous diagnostic.
- Normality — Shapiro-Wilk (n under 50) + Q-Q plot. With n of 30 or more per group, CLT provides robustness.
- Homogeneity of variance — Levene’s test. If violated, use Welch’s t-test or Welch’s ANOVA.
- Independence — Clustered data requires mixed-effects models or GEE.
- Outliers — Box plots; Cook’s distance for regression.
Tests for Comparing Groups
T-Tests: Two-Group Comparisons
- Independent samples t-test — two separate groups, different subjects.
- Paired t-test — same subjects measured twice.
- One-sample t-test — compares sample mean against a known population value.
45 patients on drug A vs 43 on placebo; mean SBP reduction compared. Shapiro-Wilk p = 0.31, Levene’s p = 0.44. Test: Independent t-test. t(86) = 3.12, p = 0.003, Cohen’s d = 0.67 (95% CI: 0.23–1.10) — medium-to-large effect.
ANOVA: Three or More Groups
Running three separate t-tests inflates the family-wise error rate to ~14%.7 Always follow significant ANOVA with Tukey’s HSD (equal n) or Bonferroni.
LDL reduction across 3 dose groups (n=30 each). Test: One-way ANOVA. F(2,87) = 8.43, p = 0.0004, η² = 0.16. Post-hoc Tukey: high vs low (p = 0.001), high vs standard (p = 0.038); low vs standard NS.
Chi-Square and Fisher’s Exact: Categorical Outcomes
Complication rates: laparoscopic (n=120, 18 events) vs open (n=115, 31 events). Test: Chi-square. χ²(1) = 5.14, p = 0.023, Cramér’s V = 0.15. Laparoscopic significantly lower (15% vs 27%).
Tests for Relationships and Associations
- Pearson’s r — two continuous, normally distributed variables. Always report r with 95% CI.
- Spearman’s rho — ordinal, non-normal, or monotonic relationships.
- Kendall’s tau — preferred over Spearman for small samples with many tied ranks.
Regression Analysis
| Outcome Type | Regression Model | Key Output |
|---|---|---|
| Continuous | Linear regression | Beta coefficient, R², 95% CI |
| Binary (yes/no) | Binary logistic regression | Odds ratio (OR), 95% CI, AUC-ROC |
| Time-to-event | Cox proportional hazards | Hazard ratio (HR), 95% CI |
| Count data | Poisson / negative binomial | Incidence rate ratio (IRR), 95% CI |
| Ordinal outcome | Ordinal logistic regression | Proportional OR, 95% CI |
Predicting 30-day readmission from age, Charlson Index, length of stay. Charlson OR = 1.42 (95% CI: 1.18–1.71, p <0.001). c-statistic = 0.74. Each additional comorbidity increased readmission odds by 42% after adjustment.
Beyond P-Values: What Journals Require
A 2026 review found subtle p-value misuse persists in top epidemiology journals.9 A complete statistical report requires: test statistic with df, exact p-value, effect size with 95% CI, and direction/magnitude of the effect.2
Statistical Software & EditVerse Services
| Software | Best For | Cost | EditVerse Can Help |
|---|---|---|---|
| R (rstatix, tidyverse) | Reproducible analysis, all test types | Free | Manuscript Stats Service |
| SPSS | Clinical research; point-and-click | Paid | Manuscript Stats Service |
| Stata | Epidemiology, panel data, survival | Paid | Advanced Statistical Analysis |
| SAS | Regulatory (FDA), large datasets | Paid / free OnDemand | Advanced Statistical Analysis |
| Python (scipy, statsmodels) | Data pipelines, ML integration | Free | Manuscript Stats Service |
Related Statistical Guides on EditVerse
- Regression vs Correlation in Clinical Research
- Propensity Score Matching
- T-tests in Clinical Trials with Effect Size
- Understanding Statistical Significance
- Sample Size Calculator Guide
FAQ
How do I choose between a parametric and non-parametric test?
Use a parametric test when your data is continuous and approximately normally distributed, or when n is 30 or more per group. Use a non-parametric test when data is ordinal, the sample is small and non-normal, or influential outliers are present.
When should I use ANOVA instead of multiple t-tests?
Always use ANOVA when comparing three or more groups. Running three pairwise t-tests at alpha = 0.05 inflates the family-wise error rate to approximately 14%. Follow a significant ANOVA with Tukey HSD or Bonferroni post-hoc tests.
What is the difference between chi-square and Fisher’s Exact Test?
Use chi-square when all expected cell counts are 5 or more. Use Fisher’s Exact Test when any expected cell count is under 5 or the sample is small. Fisher’s Exact Test is always valid for 2×2 tables.
How do I test for normality before choosing my test?
Use Shapiro-Wilk for n under 50, combined with Q-Q plot inspection. Shatz (2024, Behavior Research Methods) shows Shapiro-Wilk alone is unreliable. With n of 30 or more per group, the Central Limit Theorem provides robustness for parametric tests.
What effect size should I report with each test?
Cohen’s d for t-tests, eta-squared or omega-squared for ANOVA, r or rho for correlation, odds ratios or hazard ratios for logistic and Cox regression, R-squared for linear regression. Always include the 95% confidence interval.
What is logistic regression used for?
Binary logistic regression predicts the probability of a binary outcome from one or more predictors, outputting odds ratios (OR) with 95% CIs. If outcome prevalence exceeds 10%, consider Poisson regression with robust SEs to estimate risk ratios directly.
Can Pearson correlation control for confounders?
No. Use linear regression (continuous outcomes), logistic regression (binary), or Cox regression (survival) when other variables must be controlled.
Source Links
- Ben Suleiman A et al. Statistical errors in clinical prosthodontic publications, 2019-2024. J Prosthodont. 2026. PMC13088005.
- Strasak AM et al. Statistical errors in medical research. Swiss Med Wkly. 2007. PMC3116565.
- UCLA Statistical Methods — What Statistical Analysis Should I Use?
- Kaur P et al. Descriptive statistics. Int J Acad Med. 2018. PMC4560542.
- Shatz I. Assumption-checking rather than (just) testing. Behav Res Methods. 2024. PMC10830673.
- GraphPad — The Ultimate Guide to T-Tests.
- Kim TK, Park JH. Assumptions of t-test. Korean J Anesthesiol. 2021. PMC7941456.
- Enago Academy — How to Choose the Right Statistical Test.
- Hunter MK et al. P-values and confidence intervals in epidemiology journals. Glob Epidemiol. 2026. PMC12834938.
- Emerald Publishing — How to Choose the Right Statistical Technique.