Claude Skill

alterlab-statistical-analysis

Guided statistical analysis with hypothesis-test selection, assumption checking, effect sizes, power analysis, and APA-formatted reporting using scipy.stats, statsmodels, and pingouin (Bayesian alternatives with PyMC). Use when choosing and running the appropriate statistical tes

LLM Mart · 0 points · 2 views 0 listing impressions 0 install-command copies
Download AlterLab-IEU-AlterLab-Academic-Skills-skills_data-science_alterlab-statistical-analysis-3ab6d03.zip · 41 KB

Install

skills CLI npx skills add https://github.com/AlterLab-IEU/AlterLab-Academic-Skills/tree/main/skills/data-science/alterlab-statistical-analysis
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install alterlab-ieu-alterlab-academic-skills@llmmart
Git git clone https://github.com/AlterLab-IEU/AlterLab-Academic-Skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole alterlab-ieu/alterlab-academic-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Statistical Analysis

Overview

A systematic process for testing hypotheses and quantifying relationships. Conduct hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian analyses with assumption checks and APA reporting. For academic research.

When to Use This Skill

Use when:

  • Conducting hypothesis tests (t-tests, ANOVA, chi-square)
  • Performing regression or correlation analyses
  • Running Bayesian statistical analyses
  • Checking statistical assumptions and diagnostics
  • Calculating effect sizes and conducting power analyses
  • Reporting statistical results in APA format

Does NOT Trigger

Scenario Use Instead
Fitting a specific model class (OLS/GLM/mixed/ARIMA) with coefficient tables and residual diagnostics alterlab-statsmodels
Policing test choice: switching tests after seeing p-values, test-shopping, uncorrected families of tests alterlab-test-selection-guard
Full Bayesian or hierarchical modelling workflows (custom priors, NUTS diagnostics, LOO model comparison) alterlab-pymc
Pooling effect sizes across studies (random-effects meta-analysis, heterogeneity, funnel plots) alterlab-meta-analysis
Complex-sample survey data needing weights, strata, and design-based standard errors alterlab-survey-analysis
Predictive modelling: train/test splits, cross-validated tuning, and held-out metrics (accuracy, ROC AUC, RMSE, test R²) rather than inference alterlab-scikit-learn

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-statsmodels) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

Core Capabilities

  1. Test selection & planning — choose tests by research question and data type; a priori power analysis; multiple-comparison strategy.
  2. Assumption checking — verify normality, homogeneity, linearity; diagnostic plots; remediation when violated.
  3. Statistical testing — parametric and non-parametric tests; regression; correlation; Bayesian alternatives with Bayes Factors.
  4. Effect sizes & interpretation — appropriate effect sizes with CIs; statistical vs. practical significance.
  5. Professional reporting — APA-style reports, publication-ready figures and tables.

Workflow

SELECT a test?      → Test Selection Guide
CHECK assumptions?  → Assumption Checking
RUN analysis?       → Running Statistical Tests + references/code_examples.md
REPORT results?     → Reporting Results + references/apa_report_templates.md

Worked code for every step is in references/code_examples.md.

Test Selection Guide

Quick reference (full decision tree: references/test_selection_guide.md):

Two groups — independent + normal → independent t-test; independent + non-normal → Mann-Whitney U; paired + normal → paired t-test; paired + non-normal → Wilcoxon signed-rank; binary outcome → chi-square or Fisher's exact.

3+ groups — independent + normal → one-way ANOVA; independent + non-normal → Kruskal-Wallis; paired + normal → repeated-measures ANOVA; paired + non-normal → Friedman.

Relationships — two continuous → Pearson (normal) or Spearman (non-normal); continuous outcome + predictors → linear regression; binary outcome + predictors → logistic regression.

Bayesian alternatives exist for all of the above, giving direct probability statements, Bayes Factors, and the ability to support the null. See references/bayesian_statistics.md.

Assumption Checking

Check assumptions before interpreting results, because a violated assumption can make the reported p-value and CI wrong. Use the bundled scripts/assumption_checks.py (comprehensive_assumption_check() and targeted functions — code in references/code_examples.md). It performs outlier detection (IQR + z-score), normality testing (Shapiro-Wilk + Q-Q), homogeneity of variance (Levene + box plots), and interpretation.

When violated:

  • Normality — mild + n>30/group → proceed (robust); moderate → non-parametric; severe → transform or non-parametric.
  • Homogeneity — t-test → Welch's; ANOVA → Welch's or Brown-Forsythe; regression → robust SE or WLS.
  • Linearity (regression) — polynomial terms, transform variables, or GAM.

Detail: references/assumptions_and_diagnostics.md.

Running Statistical Tests

Primary libraries: scipy.stats (core tests), statsmodels (regression/diagnostics), pingouin (tests + effect sizes), pymc + arviz (Bayesian).

Copy-paste examples for t-test, ANOVA + post-hoc, linear regression with full diagnostics, and a Bayesian t-test: references/code_examples.md.

Effect Sizes

Effect sizes quantify magnitude; p-values only indicate existence. Always report them, with CIs. Benchmarks (guidelines only — context matters):

Test Effect Size Small Medium Large
T-test Cohen's d 0.20 0.50 0.80
ANOVA η²_p 0.01 0.06 0.14
Correlation r 0.10 0.30 0.50
Regression R² 0.02 0.13 0.26
Chi-square Cramér's V 0.07 0.21 0.35

Calculation code and CI computation: references/code_examples.md. Full guidance: references/effect_sizes_and_power.md.

Power Analysis

Run a priori power analysis to size a study before collecting data, and sensitivity analysis afterward to report the smallest detectable effect. Avoid post-hoc power. Code: references/code_examples.md. Guidance: references/effect_sizes_and_power.md.

Reporting Results

Follow APA style (references/reporting_standards.md). Report descriptive statistics (M, SD, n), test statistics (name, statistic, df, exact p), effect sizes with CIs, assumption checks, and all planned analyses.

Ready-to-adapt report paragraphs for t-test, ANOVA, multiple regression, and Bayesian analysis: references/apa_report_templates.md.

Bayesian Statistics

Consider Bayesian methods when you have prior information, want direct probability statements, have small samples or sequential data, need to quantify evidence for the null, or have a complex (hierarchical, missing-data) model. Key advantages: intuitive interpretation, evidence for the null, flexibility, full uncertainty quantification. Comprehensive guide (priors, Bayes Factors, credible intervals, convergence checks): references/bayesian_statistics.md.

Best Practices

Pre-register analyses; always check assumptions; report effect sizes with CIs; report all planned analyses including non-significant ones; distinguish statistical from practical significance; visualize before and after; check regression/ANOVA diagnostics; run sensitivity analyses; share data and code; be transparent about violations and decisions.

Common Pitfalls

P-hacking; HARKing; ignoring assumptions; confusing significance with importance; not reporting effect sizes; cherry-picking results; misinterpreting p-values; uncorrected multiple comparisons; ignoring missing-data mechanism (MCAR/MAR/MNAR); overinterpreting non-significant results.

Getting Started Checklist

  • Define research question and hypotheses
  • Determine the test (references/test_selection_guide.md)
  • Power analysis for sample size
  • Load and inspect data; check missing data and outliers
  • Verify assumptions (scripts/assumption_checks.py)
  • Run primary analysis; compute effect sizes with CIs
  • Post-hoc tests if needed (with corrections)
  • Create visualizations
  • Write results (references/reporting_standards.md)
  • Sensitivity analyses; share data and code

Index of Bundled Resources

References (references/)

  • test_selection_guide.md — decision tree for choosing a test
  • assumptions_and_diagnostics.md — checking and handling violations
  • effect_sizes_and_power.md — effect sizes and power analysis
  • bayesian_statistics.md — complete Bayesian guide
  • reporting_standards.md — APA reporting rules
  • code_examples.md — worked Python for tests, diagnostics, Bayesian, power, effect sizes
  • apa_report_templates.md — ready-to-adapt APA results paragraphs

Scripts (scripts/)

  • assumption_checks.py — automated assumption checking with visualizations (comprehensive_assumption_check, check_normality, check_homogeneity_of_variance, check_linearity, detect_outliers)

Further Reading

Key textbooks: Cohen (1988), Field (2013), Gelman & Hill (2006), Kruschke (2014). Online: APA Style Guide (https://apastyle.apa.org/), Cross Validated (stats.stackexchange.com).

Part of the AlterLab Academic Skills suite.

Files (alterlab-academic-skills)
  • evals
    • evals.json 6.7 KB
      Loading…
  • references
    • apa_report_templates.md 2.4 KB
      Loading…
    • assumptions_and_diagnostics.md 11 KB
      Loading…
    • bayesian_statistics.md 17.1 KB
      Loading…
    • code_examples.md 7 KB
      Loading…
    • effect_sizes_and_power.md 15.1 KB
      Loading…
    • reporting_standards.md 19 KB
      Loading…
    • test_selection_guide.md 5 KB
      Loading…
  • scripts
    • assumption_checks.py 15.4 KB
      Loading…
  • SKILL.md 10 KB
    ---
    name: alterlab-statistical-analysis
    description: Guided statistical analysis with hypothesis-test selection, assumption checking, effect sizes, power analysis, and APA-formatted reporting using scipy.stats, statsmodels, and pingouin (Bayesian alternatives with PyMC). Use when choosing and running the appropriate statistical test for data, verifying test assumptions, computing power/sample size, or producing APA-style results for academic research. For implementing specific models programmatically prefer alterlab-statsmodels. Part of the AlterLab Academic Skills suite.
    license: MIT
    allowed-tools: Read Write Edit Bash(python:*) Bash(uv:*)
    compatibility: No API key required. Runs locally via `uv run --python ">=3.12" --with "statsmodels>=0.14" --with "pingouin>=0.6" python analysis.py` (no install step; the interpreter pin matters because an older default Python resolves older releases). Core deps are scipy, statsmodels (>= 0.14; current 0.15), and pingouin>=0.6 (current 0.6.1; 0.6 renamed output columns, e.g. p-val to p_val, CI95% to CI95, cohen-d to cohen_d); Bayesian examples additionally need pymc and arviz.
    metadata:
        skill-author: AlterLab
        version: "1.0.1"
        last_updated: "2026-09-23"
    ---
    
    # Statistical Analysis
    
    ## Overview
    
    A systematic process for testing hypotheses and quantifying relationships. Conduct
    hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian
    analyses with assumption checks and APA reporting. For academic research.
    
    ## When to Use This Skill
    
    Use when:
    - Conducting hypothesis tests (t-tests, ANOVA, chi-square)
    - Performing regression or correlation analyses
    - Running Bayesian statistical analyses
    - Checking statistical assumptions and diagnostics
    - Calculating effect sizes and conducting power analyses
    - Reporting statistical results in APA format
    
    ### Does NOT Trigger
    
    | Scenario | Use Instead |
    |----------|-------------|
    | Fitting a specific model class (OLS/GLM/mixed/ARIMA) with coefficient tables and residual diagnostics | `alterlab-statsmodels` |
    | Policing test choice: switching tests after seeing p-values, test-shopping, uncorrected families of tests | `alterlab-test-selection-guard` |
    | Full Bayesian or hierarchical modelling workflows (custom priors, NUTS diagnostics, LOO model comparison) | `alterlab-pymc` |
    | Pooling effect sizes across studies (random-effects meta-analysis, heterogeneity, funnel plots) | `alterlab-meta-analysis` |
    | Complex-sample survey data needing weights, strata, and design-based standard errors | `alterlab-survey-analysis` |
    | Predictive modelling: train/test splits, cross-validated tuning, and held-out metrics (accuracy, ROC AUC, RMSE, test R²) rather than inference | `alterlab-scikit-learn` |
    
    **Hand-off rule.** When the request matches a row above, open your answer by naming that skill by its exact id (for example `alterlab-statsmodels`) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.
    
    ## Core Capabilities
    
    1. **Test selection & planning** — choose tests by research question and data type;
       a priori power analysis; multiple-comparison strategy.
    2. **Assumption checking** — verify normality, homogeneity, linearity; diagnostic plots;
       remediation when violated.
    3. **Statistical testing** — parametric and non-parametric tests; regression; correlation;
       Bayesian alternatives with Bayes Factors.
    4. **Effect sizes & interpretation** — appropriate effect sizes with CIs; statistical vs.
       practical significance.
    5. **Professional reporting** — APA-style reports, publication-ready figures and tables.
    
    ## Workflow
    
    ```
    SELECT a test?      → Test Selection Guide
    CHECK assumptions?  → Assumption Checking
    RUN analysis?       → Running Statistical Tests + references/code_examples.md
    REPORT results?     → Reporting Results + references/apa_report_templates.md
    ```
    
    Worked code for every step is in `references/code_examples.md`.
    
    ## Test Selection Guide
    
    Quick reference (full decision tree: `references/test_selection_guide.md`):
    
    **Two groups** — independent + normal → independent t-test; independent + non-normal →
    Mann-Whitney U; paired + normal → paired t-test; paired + non-normal → Wilcoxon
    signed-rank; binary outcome → chi-square or Fisher's exact.
    
    **3+ groups** — independent + normal → one-way ANOVA; independent + non-normal →
    Kruskal-Wallis; paired + normal → repeated-measures ANOVA; paired + non-normal → Friedman.
    
    **Relationships** — two continuous → Pearson (normal) or Spearman (non-normal); continuous
    outcome + predictors → linear regression; binary outcome + predictors → logistic regression.
    
    **Bayesian alternatives** exist for all of the above, giving direct probability statements,
    Bayes Factors, and the ability to support the null. See `references/bayesian_statistics.md`.
    
    ## Assumption Checking
    
    Check assumptions before interpreting results, because a violated assumption can make the reported p-value and CI wrong. Use the bundled
    `scripts/assumption_checks.py` (`comprehensive_assumption_check()` and targeted functions —
    code in `references/code_examples.md`). It performs outlier detection (IQR + z-score),
    normality testing (Shapiro-Wilk + Q-Q), homogeneity of variance (Levene + box plots), and
    interpretation.
    
    **When violated:**
    - *Normality* — mild + n>30/group → proceed (robust); moderate → non-parametric;
      severe → transform or non-parametric.
    - *Homogeneity* — t-test → Welch's; ANOVA → Welch's or Brown-Forsythe; regression →
      robust SE or WLS.
    - *Linearity (regression)* — polynomial terms, transform variables, or GAM.
    
    Detail: `references/assumptions_and_diagnostics.md`.
    
    ## Running Statistical Tests
    
    Primary libraries: `scipy.stats` (core tests), `statsmodels` (regression/diagnostics),
    `pingouin` (tests + effect sizes), `pymc` + `arviz` (Bayesian).
    
    Copy-paste examples for t-test, ANOVA + post-hoc, linear regression with full
    diagnostics, and a Bayesian t-test: `references/code_examples.md`.
    
    ## Effect Sizes
    
    **Effect sizes quantify magnitude; p-values only indicate existence.** Always report
    them, with CIs. Benchmarks (guidelines only — context matters):
    
    | Test | Effect Size | Small | Medium | Large |
    |------|-------------|-------|--------|-------|
    | T-test | Cohen's d | 0.20 | 0.50 | 0.80 |
    | ANOVA | η²_p | 0.01 | 0.06 | 0.14 |
    | Correlation | r | 0.10 | 0.30 | 0.50 |
    | Regression | R² | 0.02 | 0.13 | 0.26 |
    | Chi-square | Cramér's V | 0.07 | 0.21 | 0.35 |
    
    Calculation code and CI computation: `references/code_examples.md`. Full guidance:
    `references/effect_sizes_and_power.md`.
    
    ## Power Analysis
    
    Run **a priori** power analysis to size a study before collecting data, and **sensitivity
    analysis** afterward to report the smallest detectable effect. Avoid post-hoc power.
    Code: `references/code_examples.md`. Guidance: `references/effect_sizes_and_power.md`.
    
    ## Reporting Results
    
    Follow APA style (`references/reporting_standards.md`). Report descriptive statistics
    (M, SD, n), test statistics (name, statistic, df, exact p), effect sizes with CIs,
    assumption checks, and all planned analyses.
    
    Ready-to-adapt report paragraphs for t-test, ANOVA, multiple regression, and Bayesian
    analysis: `references/apa_report_templates.md`.
    
    ## Bayesian Statistics
    
    Consider Bayesian methods when you have prior information, want direct probability
    statements, have small samples or sequential data, need to quantify evidence for the
    null, or have a complex (hierarchical, missing-data) model. Key advantages: intuitive
    interpretation, evidence for the null, flexibility, full uncertainty quantification.
    Comprehensive guide (priors, Bayes Factors, credible intervals, convergence checks):
    `references/bayesian_statistics.md`.
    
    ## Best Practices
    
    Pre-register analyses; always check assumptions; report effect sizes with CIs; report all
    planned analyses including non-significant ones; distinguish statistical from practical
    significance; visualize before and after; check regression/ANOVA diagnostics; run
    sensitivity analyses; share data and code; be transparent about violations and decisions.
    
    ## Common Pitfalls
    
    P-hacking; HARKing; ignoring assumptions; confusing significance with importance; not
    reporting effect sizes; cherry-picking results; misinterpreting p-values; uncorrected
    multiple comparisons; ignoring missing-data mechanism (MCAR/MAR/MNAR); overinterpreting
    non-significant results.
    
    ## Getting Started Checklist
    
    - [ ] Define research question and hypotheses
    - [ ] Determine the test (`references/test_selection_guide.md`)
    - [ ] Power analysis for sample size
    - [ ] Load and inspect data; check missing data and outliers
    - [ ] Verify assumptions (`scripts/assumption_checks.py`)
    - [ ] Run primary analysis; compute effect sizes with CIs
    - [ ] Post-hoc tests if needed (with corrections)
    - [ ] Create visualizations
    - [ ] Write results (`references/reporting_standards.md`)
    - [ ] Sensitivity analyses; share data and code
    
    ## Index of Bundled Resources
    
    ### References (`references/`)
    - `test_selection_guide.md` — decision tree for choosing a test
    - `assumptions_and_diagnostics.md` — checking and handling violations
    - `effect_sizes_and_power.md` — effect sizes and power analysis
    - `bayesian_statistics.md` — complete Bayesian guide
    - `reporting_standards.md` — APA reporting rules
    - `code_examples.md` — worked Python for tests, diagnostics, Bayesian, power, effect sizes
    - `apa_report_templates.md` — ready-to-adapt APA results paragraphs
    
    ### Scripts (`scripts/`)
    - `assumption_checks.py` — automated assumption checking with visualizations
      (`comprehensive_assumption_check`, `check_normality`, `check_homogeneity_of_variance`,
      `check_linearity`, `detect_outliers`)
    
    ## Further Reading
    
    Key textbooks: Cohen (1988), Field (2013), Gelman & Hill (2006), Kruschke (2014).
    Online: APA Style Guide (https://apastyle.apa.org/), Cross Validated
    (stats.stackexchange.com).
    
    Part of the AlterLab Academic Skills suite.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.