数据集对比分析专家

Acknowledged. I will respond as a data analysis specialist with precise, context-aware methods across preprocessing, statistical analysis, visualization, and interpretation. To proceed efficiently and maintain accuracy, please share your task or dataset and the following details:

Required inputs

• Objective and key questions: What decisions will this analysis inform? Any hypotheses to test?
• Unit of analysis and granularity: Row meaning (e.g., transaction, user-day), time coverage, sampling frame.
• Target variable(s) and key features: Definitions, data dictionary, expected relationships.
• Data sources and joins: Primary keys, foreign keys, expected constraints, known quality issues.
• Experiment/causal design (if applicable): Randomization, assignment, exposure, inclusion/exclusion criteria.
• Metrics/KPIs: Business metrics to optimize or monitor; success thresholds.
• Context constraints: Regulatory, operational, domain-specific rules.
• Preferred environment: Python/R/SQL; ability to share sample data (schema + 10–50 rows).

Standard analysis workflow

1. Problem framing

   • Define hypotheses, metrics, unit of analysis, cohorts/segments, and time windows.
   • Identify confounders and potential sources of bias.

2. Data audit and preprocessing

   • Validate schema, types, uniqueness, referential integrity, and time coverage.
   • Handle missingness (MCAR/MAR/MNAR) using appropriate strategies: listwise deletion (if MCAR and small), simple imputation (median/mode), multiple imputation (MICE), or model-based imputations; document rationale.
   • Detect and treat outliers using robust statistics (IQR/median absolute deviation) or domain rules; consider winsorization and robust models.
   • Address duplicates, inconsistent labels, and timezone issues; standardize units and encodings.
   • Prevent target leakage by ensuring features are available at prediction time.

3. Exploratory data analysis (EDA)

   • Univariate: distributions, tails, missingness maps.
   • Bivariate/multivariate: correlations (Pearson/Spearman), mutual information, pivoted group summaries, pairwise plots.
   • Temporal: trend/seasonality, stationarity diagnostics (ADF/KPSS), lag features, cohort retention curves.

4. Statistical inference

   • Variable-type and design-appropriate tests:
     • Two-group numeric: t-test (if normal, equal variances) or Welch’s t-test; Mann–Whitney U if non-normal.
     • Multi-group numeric: ANOVA or Kruskal–Wallis; post-hoc Tukey/Holm.
     • Categorical associations: chi-square or Fisher’s exact (small counts).
     • Correlation: Pearson (linear, normal), Spearman (monotonic), partial correlations controlling for confounders.
     • Regression: linear/logistic/Poisson/negative binomial; regularization (L1/L2); robust SEs; check assumptions (linearity, homoscedasticity, multicollinearity, residual diagnostics).
   • Multiple testing control via FDR (Benjamini–Hochberg) or Bonferroni where appropriate.
   • Report effect sizes and confidence intervals, not just p-values.

5. Modeling (if predictive or forecasting)

   • Baselines: simple mean/propensity/logistic baseline; naive or seasonal naive for time series.
   • Feature engineering: domain-driven features, interactions, lags/rolling stats; avoid leakage.
   • Models:
     • Classification: logistic regression, tree-based (RF/GBM), calibrated probabilities (Platt/Isotonic).
     • Regression: linear/regularized, tree-based (GBM/XGBoost/LightGBM).
     • Time series: ARIMA/SARIMA, ETS; for complex seasonality consider TBATS; cross-validated ML with time-aware splits.
     • Panel/causal: fixed/random effects, difference-in-differences, synthetic controls; check parallel trends.
   • Evaluation:
     • Classification: ROC-AUC, PR-AUC (for imbalance), log-loss, precision/recall/F1, calibration (reliability curves), decision curves.
     • Regression: MAE, RMSE, MAPE/sMAPE (mind zero-valued targets).
     • Forecasting: sMAPE, MASE, rolling-origin validation.
   • Interpretability: SHAP/feature importance, partial dependence; stability across folds.

6. Visualization

   • Choose charts aligned with data type and message: distributions (histograms/ECDF/box/violin), relationships (scatter with CI/binned heatmaps), time series (decomposition, anomaly overlays), categorical comparisons (bar/point-range with CIs).
   • Always include axis units, uncertainty bands where relevant, and clear labeling of cohorts/time windows.

7. Results interpretation and recommendations

   • Summarize key findings with quantified effects and uncertainty.
   • Identify limitations, assumptions, sensitivity analyses, and robustness checks.
   • Translate findings into actionable steps and expected impact on KPIs.

Quality and bias considerations

• Confounding: Include known covariates, stratify or adjust via regression/propensity methods.
• Selection bias: Verify sampling frame; compare sample vs population.
• Measurement error: Assess reliability of key variables; conduct sensitivity analyses.
• Data drift: For ongoing models, monitor feature distributions and performance over time.
• Reproducibility: Version data, code, parameters; provide seeds and environment details.

Deliverables (tailored to your task)

• Cleaned dataset summary and data quality report.
• EDA visuals and statistical test results with effect sizes and CIs.
• Model artifacts (if applicable): training pipeline, metrics, interpretability outputs.
• Executive summary: findings, limitations, and recommended actions.
• Reproducible notebook/script with instructions.

To begin, please provide:

• The exact question(s) you want answered and the decision context.
• Schema or sample rows, and a brief data dictionary.
• Time range, granularity, and any inclusion/exclusion rules.
• Target variable definition (if modeling) and key KPIs.
• Any known constraints, experimental setup, or suspected issues.
• Preferred tools (Python/R/SQL) and any formatting requirements for outputs.

Once I have these, I will propose a concise analysis plan and proceed with rigorous, verifiable results.

已就緒。為確保分析高效、準確並可重現，請提供以下最小必要資訊與樣本資料。收到後我將基於您的目標設計嚴謹的預處理、統計分析、可視化與結果解讀流程。

請提供

• 分析目標與決策問題：明確定義要回答的問題、成功指標（KPI/指標）、期望輸出形式（圖表、指標表、模型、建議）。
• 資料來源與結構：資料表清單、欄位名稱、資料型別（數值/類別/時間）、主鍵與關聯（外鍵）、單位與時區。
• 時間範圍與抽樣：分析期間、是否滾動視窗、抽樣策略（若資料量大）。
• 定義與口徑：指標口徑（如「活躍用戶」定義）、去重規則、缺失值處理原則、異常值處理原則。
• 統計或模型需求：需要描述統計、推論（假設檢定/置信區間）、因果推斷、預測/分類或分群。
• 約束與環境：可用工具（Python/R/SQL）、效能限制、資料存取規範與隱私要求。
• 代表性樣本：每張表提供10–50行樣本（可匿名），或一份可重現的匯出檔（CSV/Parquet），以便快速驗證欄位與分佈。

預期分析流程（可依任務調整）

1. 資料品質與預處理

• 結構檢查：主鍵唯一性、關聯完整性、重複列與重複事件。
• 缺失與異常：缺失模式（MCAR/MAR/MNAR）分析；極端值與錯誤單位/時區；類別稀疏度。
• 一致性：時間戳對齊、事件去重、口徑統一（如時區、週期、轉換漏斗邏輯）。
• 特徵工程：衍生指標（轉化率、留存、客單價、週期聚合）、編碼（類別/日期）、尺度轉換。

2. 探索性分析（EDA）

• 分佈與集中趨勢：直方圖、箱型圖、分位數、偏態與峰度。
• 關聯性：皮爾森/史匹爾曼相關、交叉表、群組對比（分群、分渠道、分時段）。
• 漏斗與路徑：步驟轉化率、瓶頸定位、用戶行為序列。

3. 推論與建模（視需求）

• 組間對比：均值/比例檢定、效果大小（Cohen’s d）、多重比較校正。
• 時序分析：季節性/週期性分解、異常偵測、干預分析。
• 預測/分類：基準模型→正則化→交叉驗證；特徵重要度與穩健性。
• 因果設計：傾向分數、雙重差分、斷點回歸（若可行且滿足假設）。

4. 可視化與交付

• 圖表：分佈、時間序列、分群對比、漏斗、地圖（若有地域）。
• 指標表：核心KPI、子維度拆解、置信區間。
• 建議：行動建議與風險；敏感度分析；可重現腳本或筆記本。

風險與精度控制

• 明確口徑與時間對齊，避免因定義差異造成偏差。
• 報告統計不確定性（置信區間、p值、效果大小）與樣本量充足性。
• 對多次比較進行校正；避免資料探勘式誤報。
• 匯報限制與假設，避免過度延伸結論。

若您已準備好資料與問題，請按上述清單提供；我將以技術寫作風格回覆，聚焦準確的分析、可視化與解讀，并給出可操作的結論與建議。