解释p值的意义

以下内容解释当 p 值等于 0.05 时的统计含义与使用要点。

一、定义

• p 值：在原假设（H0）为真且模型假设成立的前提下，统计量取到“与观测结果同样或更极端”的概率。
• 形式化：p = P(T ≥ t_obs | H0)（单侧；双侧检验为相应的双尾概率），其中 T 为检验统计量，t_obs 为观测值。

二、当 p = 0.05 时的含义

• 若 H0 为真，得到当前或更极端数据的概率为 5%。
• 这表示对 H0 的证据强度处于“临界”水平：数据对偏离 H0 提供了有限证据，但证据并不强。

三、与显著性水平 α 的关系

• 设定 α = 0.05：
  • p ≤ 0.05：拒绝 H0（控制长期 I 类错误率不超过 5%，在检验假设和模型正确时）。
  • p > 0.05：不拒绝 H0（并非“接受 H0”）。
• p 值与 95% 置信区间的对应关系（双侧、同模型）：若 p < 0.05，95% CI 不包含原假设值；反之亦然。

四、常见误解澄清

• p = 0.05 不是“原假设为真的概率为 5%”。
• 也不是“结果由随机机会造成的概率为 5%”。
• 不是本次结论出错的概率；I 类错误率指在无限重复实验下的长期比例。
• 不是效应大小或实际重要性的度量；小效应在大样本下也可能得到小 p 值。
• p = 0.049 与 0.051 实质差异很小，避免“悬崖式”解释。

五、前提与限制

• 依赖检验前提：模型正确性（分布、线性/方差齐性等）、抽样独立性、实验设计与分析方案预先设定。
• 多重比较：在多次检验下，单一阈值 α = 0.05 会放大总体 I 类错误，应进行校正（如 Bonferroni、FDR）。
• 检验功效：在样本量不足或噪声较大时，p 值可能偏大而无法发现真实效应（II 类错误风险）。
• 结果可重复性：p = 0.05 不保证重复实验仍显著。

六、实践建议

• 报告：给出精确 p 值、效应量及其置信区间，并描述检验类型与前提检验结果。
• 决策：在预先指定的 α 下进行判断，避免事后调整阈值；在多重检验中控制家族错误率或 FDR。
• 解释：将统计显著性与实际意义、领域基准、先验知识结合；避免将边界性的 p = 0.05 作为强证据。
• 稳健性：进行敏感性分析（不同模型假设、协变量集、稳健标准误）以评估结论稳定性。

Meaning of p = 0.001

Definition

• The p-value is the probability, assuming the null hypothesis (H0) and all model assumptions are correct, of obtaining a test statistic at least as extreme as the one observed.
• p = 0.001 means that, if H0 is true, results as extreme as the observed would occur by chance about 0.1% of the time under the specified test (including its tail(s)).

Interpretation when p = 0.001

• Evidence against H0: Very strong evidence against the null hypothesis under the assumed model.
• Decision at common α thresholds:
  • At α = 0.05 or α = 0.01: reject H0.
  • At α = 0.001: reject H0 if using the rule p ≤ α and α was pre-specified.
• Long-run error control: If you used α = 0.001 as your significance threshold in many identical studies where H0 is true, about 0.1% of rejections would be false positives.
• Relation to confidence intervals: In a two-sided test aligned with a CI, p = 0.001 implies that the 99.9% CI would exclude the null value (assuming identical model/assumptions).

What p = 0.001 does not mean

• Not the probability that H0 is true or false.
• Not the probability that the result is a false positive given the data.
• Not a measure of effect size or practical importance.
• Not a replication probability.
• Not valid evidence if model assumptions are violated or if analyses were data-driven without proper control.

Assumptions and context that affect interpretation

• Correct model specification and assumptions (e.g., independence, distributional form, variance structure).
• Pre-specification of:
  • The null and alternative hypotheses (one-sided vs two-sided).
  • The test statistic and analysis plan.
  • Significance level (α).
• Multiple testing: If many hypotheses are tested, adjust for multiplicity (e.g., Bonferroni, Holm, FDR). A raw p = 0.001 may no longer be ≤ adjusted α after correction.
• Sample size and power: Large samples can yield very small p-values for trivially small effects; small samples may yield p = 0.001 only with a large observed effect or low variability.

Practical implications

• Treat p = 0.001 as strong statistical evidence against H0, conditional on assumptions.
• Evaluate effect size and its CI to assess practical/clinical significance.
• Consider prior evidence and the study design; avoid post hoc thresholds or selective reporting.

Reporting recommendations

• Report: the exact p-value (0.001), test type, sidedness, effect size (with units), confidence interval, sample size, model assumptions, and any multiplicity adjustments.
• Example: “Two-sided t-test of mean difference = 0: n = 120, mean difference = 0.42 (95% CI: 0.21 to 0.63), p = 0.001; assumptions checked; no multiple-testing adjustments.”