制定数据收集计划

Data Collection Plan: The Effect of Blended Learning on Academic Self-Efficacy

1. Purpose and Research Questions

• Purpose: To estimate the causal effect of blended learning (BL) on students’ academic self-efficacy (ASE) and to document implementation fidelity and contextual moderators.
• Primary question: Does participation in a well-specified BL course increase ASE relative to traditional face-to-face instruction?
• Secondary questions:
  • Do effects vary by baseline ASE, prior achievement, discipline, or gender?
  • Are BL effects mediated by mastery experiences and instructional presence?
  • How do implementation fidelity and engagement relate to ASE change?

Rationale: BL may enhance ASE by increasing mastery opportunities, feedback, and learner control (Bandura, 1997; Graham, 2006; Means et al., 2013).

2. Design Overview

• Preferred design: Cluster randomized controlled trial (cRCT) at the course-section level to minimize contamination (randomize sections to BL vs. business-as-usual face-to-face).
• Alternative (if randomization not feasible): Quasi-experimental matched comparison with propensity score methods and difference-in-differences using baseline ASE, prior achievement, demographics, and prior online experience as covariates (Murnane & Willett principles of causal inference; see also WWC standards).
• Mixed-methods concurrent design: Quantitative (surveys, LMS logs, administrative data) with qualitative interviews/focus groups for triangulation and explanatory depth (Creswell & Plano Clark, 2017).

3. Setting, Sampling, and Participants

• Setting: Multiple undergraduate gateway courses (e.g., introductory psychology, biology, statistics) across 2–4 institutions to enhance generalizability (Graham, 2006).
• Units:
  • Clusters: Course sections (target 40–80 sections total, balanced across arms and disciplines).
  • Students: All enrolled students in selected sections; anticipated n per section = 25–40.
  • Instructors: Faculty teaching participating sections.
• Inclusion criteria: Degree-seeking undergraduates enrolled at census; instructors willing to implement the specified BL model or the standard face-to-face format.
• Exclusion criteria: Sections taught by graduate assistants without training support; courses with existing extensive online components in the control arm.

4. Intervention and Comparison (for Fidelity Anchoring)

• BL treatment: Pre-specified blended model with defined dosage (e.g., 30–50% online), structured weekly online modules (readings, quizzes, discussion), and in-person active learning sessions. Alignment documented via a BL design template and a checklist derived from established blended/course quality frameworks (e.g., Community of Inquiry presence indicators; Arbaugh et al., 2008; Graham et al., 2013).
• Control: Business-as-usual face-to-face delivery without systematic online components beyond standard LMS posting.

5. Outcome Measures Primary outcome: Academic self-efficacy

• Instrument: MSLQ Self-Efficacy for Learning and Performance subscale (8 items; 1–7 Likert) administered at baseline (T0), midterm (T1), and end of term (T2) (Pintrich et al., 1991; Pintrich et al., 1993).
• Evidence: Consistently high internal consistency for college samples (typically α ≈ .90) and demonstrated predictive validity for achievement (Pintrich et al., 1993).

Secondary/auxiliary measures (for mechanism, covariate adjustment, and sensitivity)

• Sources of self-efficacy (mastery experiences, vicarious experiences, social persuasion, physiological states), adapted from domain-appropriate scales following Bandura’s construction guidelines (Bandura, 2006; Usher & Pajares, 2009).
• Teaching, social, and cognitive presence (CoI survey; Arbaugh et al., 2008) to capture instructional/learning environment characteristics associated with BL.
• Prior achievement: Cumulative GPA or standardized placement scores; baseline course diagnostic if available.
• Engagement/effort: LMS activity metrics (time-on-task, resource views, assignment submission patterns) and brief self-report engagement scale.
• Demographics: Age, gender, major/discipline, first-generation status, prior online course experience.

6. Implementation Fidelity and Exposure

• Instructor-reported adherence: Weekly implementation logs aligned to the BL design template (dosage of online components, use of active learning).
• Independent observations: Structured observations of in-person sessions using a validated active-learning checklist.
• LMS analytics: Actual BL dosage (e.g., completion of online modules, discussion participation, quiz attempts).
• Fidelity rubric: Global fidelity scores synthesized from logs, observations, and analytics (O’Donnell, 2008).

7. Timing and Procedures

• Pre-semester (T−1): Instructor recruitment; randomization at section level; training for BL instructors; pilot testing of instruments; LMS instrumentation.
• Week 1–2 (T0): Baseline student survey (ASE, demographics, prior online experience), consent, and retrieval of prior GPA. Randomization concealed from analysts; treatment known to implementers.
• Midterm (T1): Short ASE assessment, CoI survey, and engagement check to examine trajectories and reduce common method bias via temporal separation (Podsakoff et al., 2003).
• End of term (T2): Posttest ASE, CoI, course grade collection; instructor fidelity summaries; LMS data export.
• Postterm (T3, optional): Follow-up ASE 6–8 weeks later to assess persistence.
• Qualitative sampling: Purposive subsample (e.g., n ≈ 20–30 students per condition across disciplines; 8–12 instructors) for semi-structured interviews near T2 to explain quantitative patterns (Creswell & Plano Clark, 2017).

8. Data Quality Assurance

• Pilot: Cognitive interviews with 8–12 students to ensure clarity and contextual fit of items; small pilot (n ≈ 60–80) to examine reliability and preliminary factor structure.
• Measurement equivalence: Test measurement invariance (configural, metric, scalar) for the MSLQ ASE subscale across conditions and time points before estimating effects (Putnick & Bornstein, 2016).
• Administration controls: Uniform survey windows, standardized reminders, proctoring in class where feasible to maximize response rates; incentives (e.g., small course credit or raffle) approved by IRB.
• Nonresponse management: Track response propensity; implement targeted reminders; document reasons for attrition.

9. Sample Size and Power Guidance

• Plan for a cRCT with students nested in sections. Use established software (e.g., Optimal Design) with plausible assumptions: small effect size (d = 0.20–0.25), intraclass correlation at section level ICC ≈ .03–.07 for psychosocial outcomes, covariate R2 ≈ .40 from baseline ASE and GPA. Aim for at least 30–40 sections per arm with 25–35 students each to achieve ≈ .80 power for small effects (Hedges & Rhoads, 2010; Raudenbush & Bryk, 2002). Final targets should be set via a formal power analysis with local ICC estimates.

10. Data Management and Security

• Unique study IDs linking surveys, LMS logs, and grades; no storage of direct identifiers with analytic datasets.
• Secure, encrypted storage on institutional servers; access controls; audit logs.
• Codebook, metadata, and version control (e.g., Git with restricted access).
• Pre-registration of hypotheses, primary outcomes, and analysis plan (e.g., OSF) before data collection.
• Data sharing: De-identified datasets and instruments archived following FAIR principles when permitted (Wilkinson et al., 2016).

11. Ethical Considerations

• IRB approval; informed consent specifying voluntary participation, data uses, and withdrawal rights.
• FERPA-compliant handling of educational records; minimal-risk classification anticipated.
• Mitigation of risks: Anonymized reporting; instructor training to ensure equitable learning opportunities in both arms.

12. Minimizing Bias and Ensuring Internal Validity

• Randomization by an independent coordinator; blocked by course and instructor to balance prior outcomes.
• Baseline measures to adjust for any residual imbalance.
• Analyst blinding to treatment during data cleaning and pre-specification of models.
• Multiple data sources (surveys, logs, grades) and temporal separation to reduce common method variance (Podsakoff et al., 2003).
• Monitoring protocol deviations and documenting contamination.

13. Analysis-Linked Data Needs (to inform collection)

• Primary impact: Change in ASE (T2 minus T0) estimated with multilevel modeling (students nested in sections), adjusting for baseline ASE and covariates (Raudenbush & Bryk, 2002).
• Moderation: Interaction terms for baseline ASE, discipline, gender.
• Mediation (exploratory): Mastery experiences and teaching presence measured at T1 and T2 (Bandura, 1997; Arbaugh et al., 2008).
• Sensitivity: Per-protocol and complier-average causal effect analyses using fidelity indices; missing data handled via FIML or multiple imputation under MAR assumptions (Enders, 2010).

14. Reporting and Documentation

• Document recruitment, participation flow, response rates, and attrition by arm (CONSORT for cRCTs adapted to education).
• Report reliability (Cronbach’s alpha, omega), CFA/invariance results, and fidelity statistics.
• Provide a transparent account of deviations from the pre-registered plan and robustness checks.

References

• Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C., & Swan, K. P. (2008). Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample. The Internet and Higher Education, 11(3–4), 133–136.
• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
• Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. Pajares & T. Urdan (Eds.), Self-efficacy beliefs of adolescents (pp. 307–337). Information Age.
• Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). SAGE.
• Enders, C. K. (2010). Applied missing data analysis. Guilford Press.
• Graham, C. R. (2006). Blended learning systems: Definition, current trends, and future directions. In C. J. Bonk & C. R. Graham (Eds.), The handbook of blended learning (pp. 3–21). Pfeiffer.
• Graham, C. R., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4–14.
• Hedges, L. V., & Rhoads, C. (2010). Statistical power analysis in education research (NCEE 2010-4017). U.S. Department of Education.
• Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record, 115(3), 1–47.
• O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes. Review of Educational Research, 78(1), 33–84.
• Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). University of Michigan.
• Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the MSLQ. Educational and Psychological Measurement, 53(3), 801–813.
• Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research. Journal of Applied Psychology, 88(5), 879–903.
• Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance in the social and behavioral sciences. Child Development Perspectives, 10(1), 29–34.
• Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2nd ed.). SAGE.
• Usher, E. L., & Pajares, F. (2009). Sources of self-efficacy in mathematics: A validation study. Contemporary Educational Psychology, 34(1), 89–101.
• Wilkinson, M. D., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.

Note: If a quasi-experimental design is required, add a data element for teacher/course matching variables (e.g., historical grading distributions, class size, instructor experience) to strengthen the propensity model and include a clear overlap/diagnostics protocol before estimating effects.

以下为“教师培训对教学质量的前后测评估”之数据收集计划。方案以因果推断与测量严谨性为核心，强调多源数据、标准化工具、信度效度保障与实施忠实度监测，以确保对培训影响的估计准确、可复现与可推广。

一、研究目的与评估框架

• 研究目的：评估特定教师培训项目对课堂教学质量的影响，并探究影响机制（如培训实施忠实度、教师自我效能感变化）与异质性效应（年资、学科、学段）。
• 核心假设：与对照组相比，接受培训的教师在课堂教学质量观察评分、学生感知教学质量、与培训内容相关的教师知识/技能测验上表现更优，并在随访期保持或扩大效果；学生学习投入与学业成绩存在正向但相对较小的效应。
• 评估原则：三角互证（课堂观察+学生调查+客观作业/测评+教师自陈）、多时点测量（基线/后测/随访）、实施过程数据并行采集（忠实度与质量）。

二、研究设计与样本

• 设计类型：
  1. 优先方案：集群随机对照试验（学校或教研组为单位），T0基线测，培训后T1短期测（培训结束后2–4周）、T2中期随访（3–4个月）、T3远期随访（6–9个月）。
  2. 次优方案：准实验差异中的差异（DiD），基线等值检验并应用倾向评分加权/匹配与协变量调整。
• 样本与分层：覆盖不同学段与学科，按学段×学科×地区分层抽样，保证外部效度；每校抽取目标教师全体或随机抽取；学生调查与成绩数据来自所教班级。
• 规模与功效（示例，需据真实参数精算）：以教师观察评分为主结果、期望标准化效应d≈0.30、显著性α=0.05、检验效能80%、两层结构（教师嵌套学校，学校内相关ICC≈0.05–0.10），若每校10名目标教师，则每组需约10–15所学校（总教师数约200–300）更为稳健；建议使用PowerUpR或Optimal Design按实际ICC、变异度和失访率进行功效分析（Spybrook等, 2011）。
• 基线等值：按WWC规范检验标准化差异；若0.05<|SMD|≤0.25，后续模型需协变量调整（What Works Clearinghouse, 2020）。

三、指标体系与测量工具 A. 主要结果（课堂教学质量）

• 课堂观察量表（择一为主、可辅以第二量表做稳健性检验）：
  1. CLASS（学前、小学、初高中版本）：情感支持、课堂组织、教学支持维度，适合跨学段评估（Pianta等）。
  2. Danielson《教学框架》：计划与准备、课堂环境、教学、专业责任（Danielson, 2013）。
  3. 若培训聚焦学科教学，可选学科专向量表，如数学教学质量（MQI）（Hill等）。
• 观察设计：每名教师每时点采集2–3节常态化课堂视频（每节35–45分钟），两名独立评分员盲评；至少20%样本双评，定期漂移校准；采用多面向Rasch或一般化理论分解误差并进行评分者严厉度修正（Shavelson & Webb, 1991）。

B. 次要结果

• 学生感知教学质量：标准化学生问卷（如Tripod 7Cs或同类经验证问卷），班级层面聚合分数；每班至少20名学生问卷以提升信度（MET项目证据；Kane等）。
• 学生学习结果：与课程一致的标准化测验或高质量课程基测/期末分；如使用自编测验，需设置锚题实现前后等值并进行项目反应理论（IRT）标定。
• 教师知识与技能（与培训内容对齐）：如学科教学法/内容知识测验；采用既有经验证题库或自编并进行专家审查、试测、信度分析与等值。
• 教师自我效能感：教师教学效能量表TSES（长/短版）（Tschannen-Moran & Hoy, 2001），经跨文化适配与测量等值检验。
• 教学产出与作业质量：收集教案、作业与评阅样本，采用经验证评分规准（如IQA框架）进行盲评。
• 学生行为性投入：短式课堂投入观察清单或经验采样（在有条件时）。

C. 实施过程与忠实度

• 指标维度：到课率/时长（剂量）、活动完成度（依从/一致性）、培训者素质与互动质量、实践反馈与同伴交流强度（参照Desimone, 2009；Century等, 2010）。
• 工具：签到与时长记录、培训者与教练日志、抽样旁听评分表、教师反思札记与作业完成度量表、访谈/焦点小组。

四、数据收集流程与时间安排

• T0基线（培训前2–4周）：
  1. 教师：课堂视频2–3节；知识/技能测验；TSES；基本背景（年资、最高学历、职称、班额、课时量等）。
  2. 学生：感知教学质量问卷；可用的最近一次标准化学业数据或组织基线测。
  3. 学校：资源与管理特征（规模、师生比、教学常规、地区属性）。
• 培训实施期：过程数据滚动采集（每次活动签到、日志、随堂质性记录）；教练/督导定期记录与抽检。
• T1后测（培训结束后2–4周）：
  1. 教师：课堂视频2–3节；知识/技能测验平行版或等值卷；TSES。
  2. 学生：感知教学质量问卷；单元或期末标准化测验。
  3. 实施忠实度：完整汇总并质控。
• T2/T3随访（3–4月/6–9月）：同T1的缩减版，至少包含课堂视频与学生问卷，优先获取标准化成绩并评估维持效应。
• 日程与现场组织：避免考试周与节假日；随机排列拍摄顺序；统一摄录规范（机位、收音、取景）；建立异常事件报告与替代采集预案。

五、质量控制与信度效度保障

• 工具选择与跨文化适配：优先采用经验证量表；若需翻译，采用“翻译—回译—专家评审—认知访谈—小样本试测”的流程（Beaton等, 2000），并开展验证性因素分析与测量等值检验（跨时点与组别）。
• 评分员培训与校准：
  1. 培训天数与合格标准：依据量表官方要求（如CLASS需认证）；考核通过方可上岗。
  2. 盲评与漂移控制：评分员对组别与时点双盲；每月进行共评分与阈值校准；计算ICC/加权Kappa，目标≥0.70。
  3. 一般化理论/多面向Rasch：估计任务数、评分员数与时长对信度的贡献，优化采样设计以达G系数≥0.70。
• 测验质量：
  1. 前后测等值：设置锚题；用IRT或同等化方法链接分数，控制天花板/地板效应。
  2. 内部一致性：Cronbach’s α或ω≥0.70；条目反应特征检查与偏差项处理。
• 学生调查实施：
  1. 统一监考与电子化施测；最小化社会期许偏差与教师干预。
  2. 班级层面聚合并报告设计效应；对小样本班级设最小样本阈值（建议≥15–20）。
• 盲法与偏差控制：观察与评分全程盲法；对照组提供延迟培训以提高参与同意率；预防霍桑效应与补偿性竞争。
• 数据完整性与缺失：
  1. 监控关键KPI（视频完成率、双评比例、问卷有效率、测验缺失率）。
  2. 设立追补与替代采样机制；记录缺失原因；分析阶段使用多重插补并报告敏感性分析。

六、数据管理与伦理合规

• 审批与同意：通过机构伦理审查；获取教师与学生家长（或监护人）知情同意/同意豁免；对视频与问卷用途、保存期限、去标识化与撤回权利进行明示。
• 隐私与安全：分离身份信息与研究数据；视频与PII加密存储、限权访问；建立数据保留与销毁计划；遵循《教育和心理测量标准》(AERA/APA/NCME, 2014)。
• 数据治理：编制数据字典与版本控制；双人复核关键变量；日志记录数据变更；预设异常值与逻辑校验规则。

七、预注册与分析准备（与数据收集直接相关的要点）

• 预注册：在OSF登记研究与前分析计划，明确定义主次结果、时点、排除准则与协变量集。
• 统计框架（概述）：多层模型（学生/课堂/教师/学校），对基线值与关键协变量调整；组别×时点交互项为主效应；聚类稳健标准误；在有评分者的结果中使用多面向Rasch或多层模型控制评分者效应。
• 异质性与机制：预先定义分层（年资、学科、基线水平、学校特征）；以实施忠实度作为中介/调节进行机制探索。
• 报告规范：遵循WWC与透明报告原则，公开材料与（去标识化）数据字典。

八、关键操作细节清单（供现场执行对照）

• 课堂视频：每位教师每时点2–3节；两名盲评评分员；≥20%双评；每月漂移校准一次。
• 学生调查：同一周内完成；每班≥20份有效问卷为目标；统一施测说明。
• 测验：平行卷或IRT等值；锚题比例建议≥20%；设回收与补测窗口。
• 忠实度：每次培训全程签到（时长到分钟）、内容覆盖核对表、教练/讲师日志、抽样旁听评分；形成每位教师的剂量与质量指数组合。
• 质量阈值：观察评分ICC≥0.70；问卷α≥0.80（总分）；测验信息函数覆盖目标能力区间；缺失率控制在<10%并记录原因。

参考文献（示例，按APA风格）

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing.
• Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191.
• Century, J., Rudnick, M., & Freeman, C. (2010). A framework for measuring fidelity of implementation. American Journal of Evaluation, 31(2), 199–218.
• Danielson, C. (2013). The framework for teaching evaluation instrument. Danielson Group.
• Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199.
• Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have we identified effective teachers? Bill & Melinda Gates Foundation MET Project.
• Kraft, M. A., Blazar, D., & Hogan, D. (2018). The effect of teacher coaching on instruction and achievement. Review of Educational Research, 88(4), 547–588.
• Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System (CLASS) manual. Paul H. Brookes.
• Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. SAGE.
• Spybrook, J., et al. (2011). Optimal design for longitudinal and multilevel research. In S. H. Cole (Ed.), The Oxford handbook of quantitative methods.
• Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783–805.
• What Works Clearinghouse. (2020). Procedures and standards handbook (Version 4.1/5.0). Institute of Education Sciences.

注：具体工具的中文版本与使用授权需在项目启动前确认；若进行本土化适配，须完成系统的翻译、等值与信效度检验流程，并在预注册中明确主结果的操作化定义与计分方案。上述采样规模、ICC与效应量为常见取值的示意，实际研究应基于先导研究与现地参数进行功效与设计优化。