制定假设的统计分析

统计假设（围绕“新广告文案是否提升注册转化率”）

- 指标定义：注册转化率（CVR）= 完成注册的用户数 / 来自该广告的有效点击的唯一用户数。若您以曝光为基准，请将分母替换为有效曝光的唯一用户数，但点击基准更贴近注册行为。

- 主假设（单侧，关注提升）：
  - H0（零假设）：p_new ≤ p_control（新文案的注册转化率不高于旧文案）
  - H1（备择假设）：p_new > p_control（新文案的注册转化率高于旧文案）

- 业务阈值版本（如需设定最小可接受提升）：
  - H0：p_new − p_control ≤ δ
  - H1：p_new − p_control > δ
  - 其中 δ 为您设定的最小提升幅度（如绝对提升0.5个百分点或相对提升5%）。

- 备注（如仅验证是否有差异而非提升方向，可用双侧检验）：
  - H0：p_new = p_control
  - H1：p_new ≠ p_control

结论判定标准：在同等条件下（随机分流、相同受众与归因口径）拒绝零假设，即可判定新广告文案提升注册转化率。

Objective
Assess whether the new guidance process increases the novice task completion rate compared with the current process.

Operational definitions
- Population: Novice users (e.g., first-time users or those with ≤1 prior completed task).
- Primary outcome: Task completion rate within a defined window (e.g., completion of the first assigned task within 7 days).
- Metric: Proportion of novices who complete the task (user-level binary outcome).

Statistical hypotheses
- Null (H0): The new guidance process does not increase the novice task completion rate versus control (p_new ≤ p_control).
- Alternative (H1): The new guidance process increases the novice task completion rate (p_new > p_control).

Success criterion (practical significance)
- Minimum detectable effect: Absolute increase of at least 5 percentage points in completion rate (e.g., from 40% to ≥45%), in addition to statistical significance.

Design and analysis plan
- Design: Randomized A/B test at the user level; stratify by key factors (e.g., platform, geography), and block by cohort start date to control time effects. Prevent contamination (e.g., one user sees only one process).
- Primary test: One-sided two-proportion z-test at α = 0.05.
- Estimation: Report risk difference and 95% CI; optionally use logistic regression with covariate adjustment for precision.
- Unit of analysis: User-level indicator to avoid within-user clustering issues.

Sample size illustration
- Assumptions: Baseline completion p_control = 0.40; target p_new = 0.46 (Δ = 0.06); α = 0.05 (one-sided); power = 0.80.
- Required per group ≈ [(1.645 + 0.842)² × (0.40×0.60 + 0.46×0.54)] / 0.06² ≈ 840 users per arm.
- Inflate for expected exclusions/attrition (e.g., 10–15%).

Decision rule
Reject H0 and conclude improvement if:
- The observed difference (p_new − p_control) is statistically significant (p < 0.05, one-sided), and
- The effect meets the practical threshold (≥5 percentage points) or meets pre-specified business criteria.

Key risks and controls
- Seasonality/launch timing: Use blocking and run until sample size met.
- Imbalance: Check covariate balance post-randomization.
- Measurement: Predefine novice criteria and completion window; exclude invalid users (bots/duplicates).
- Analysis integrity: Pre-register hypotheses and analysis; avoid peeking and optional stopping.

Conclusion
Hypothesis: The new guidance process increases novice task completion rates. Test via a randomized A/B experiment using a one-sided proportion test, with a practical improvement threshold of at least 5 percentage points and an estimated sample size of ~840 users per group under typical assumptions.

研究问题
不同获客渠道的用户首日留存率是否存在显著差异？

指标定义
首日留存率（D1）= 某渠道在统计日新增用户中，次日仍活跃的用户数 / 该渠道新增用户总数。所有渠道需使用一致的留存口径与时间窗口。

统计假设（总体）
- 原假设 H0：各渠道的首日留存率相等（p1 = p2 = … = pk）。
- 备择假设 H1：至少有一个渠道的首日留存率不同（∃ i,j 使 pi ≠ pj）。

检验方案
- 总体检验：构建“渠道 × 是否首日留存”的列联表，使用卡方检验（α=0.05，双侧）。同时报告效应量（Cramér’s V）。
- 事后比较（在拒绝H0时）：对渠道两两比较
  - 假设：H0_ij：pi = pj；H1_ij：pi ≠ pj。
  - 方法：两比例 z 检验（样本充足）或 Fisher 精确检验（任一格期望频数<5）。
  - 多重比较调整：控制假阳性，建议 Benjamini–Hochberg（FDR=0.05）或保守的 Bonferroni。
  - 输出：差异的绝对值（Δ=pi−pj）、相对比值（风险比或胜算比）、95%置信区间与p值。

关键假设与数据要求
- 每位用户仅归因于一个渠道；样本独立。
- 渠道归因准确且无重复、机器人流量干扰（必要时做数据清洗）。
- 相同统计期与版本/市场环境，确保可比性。
- 若需控制混杂（如国家、设备、版本、投放日期等），采用逻辑回归：Retained ~ Channel + Covariates，并检验渠道系数是否显著（βi=0 vs βi≠0）。

判定与报告
- 若总体卡方检验 p<0.05：判定渠道间首日留存存在显著差异；随后通过事后比较定位差异渠道，并报告效应量与区间估计。
- 若 p≥0.05：未观察到显著差异，应报告统计功效与样本量是否足够，以避免否定之误。