设计模块自评问题

以下问题面向“测量与信度”模块的自我评估，覆盖概念理解、方法选择、计算应用与研究设计。每题均给出作答要点或参考答案要点，并以权威文献为依据。

1. 单项选择题（概念） 题干：在经典测验理论（CTT）框架下，下列哪一表述最准确地定义了“信度”？ A. 测验得分与效标之间的相关性 B. 观测分数方差中可归因于真分数方差的比例 C. 测验条目之间的平均相关 D. 测验得分在不同群体间的稳定性 参考要点：B 为正确答案。CTT 将观测分数视为真分数与随机误差之和，信度定义为 Var(T)/Var(X)，并受情境与样本影响（AERA, APA, & NCME, 2014；Lord & Novick, 1968）。

2. 计算题（标准测量误差与置信区间） 题干：某量表在目标样本上的总分标准差 SD=10，内部一致性（例如 ω_total 或 α）为 0.84。某学习者观测分数为 70。计算：

• 标准测量误差 SEM；
• 该学习者真分数的 95% 置信区间。 参考答案要点：
• SEM = SD × sqrt(1 − ρ_xx) = 10 × sqrt(1 − 0.84) = 10 × 0.4 = 4。
• 95% 置信区间：X ± 1.96 × SEM = 70 ± 7.84，即约 [62.16, 77.84]。 注：该区间在正态误差与所用信度估计适配的前提下成立（AERA et al., 2014；Lord & Novick, 1968）。

3. 情境判断与报告（内在一致性与维度性） 题干：一份 20 题五级评分量表在样本上 α=0.92；探索性因子分析提示两个潜在因子。是否可据此声称该量表“高度可靠且单维”？应如何改进报告与证据链？ 参考要点：

• 不能仅凭 α=0.92 断言单维或高质量信度。α 假设近似等值性（tau-equivalence），对条目数量敏感，且对多维性缺乏诊断力（Cronbach, 1951；Sijtsma, 2009）。
• 建议补充：基于同一维度模型的 ω_total 与（若假设层级结构）ω_hierarchical；验证性因子分析（CFA）检验单维拟合；必要时报告分量表的内部一致性（McDonald, 1999；Revelle & Zinbarg, 2009；Dunn, Baguley, & Brunsden, 2014）。
• 说明信度为“分数在特定用途与群体上的属性”，避免将 α 作为唯一或用途无关的指标（AERA et al., 2014）。

4. 研究设计题（评分者信度） 题干：你需评定学生写作表现，采用四点序位量规，3 名评分者同时评分。选择合适的评分者信度指标并简述估计与改进步骤。 参考要点：

• 序位数据：优先使用加权 Kappa（建议二次权重）评估两两评分者一致性；若将评分近似为连续并关注“绝对一致”，可用双向随机效应、绝对一致型 ICC（Shrout & Fleiss 的 ICC(2,1)；或 McGraw & Wong 的 ICC(A,1)/(A,k) 视单个评分或平均分）（Cohen, 1968；Shrout & Fleiss, 1979；McGraw & Wong, 1996）。
• 若关注多来源误差（评分者、题目等），可采用广义理论（G-theory）估计 G 系数/Φ 系数，并据此优化抽样与设计（Shavelson & Webb, 1991）。
• 改进策略：评分者培训与校准、锚例与评分指南精细化、盲评与复核、必要时增加评分者或题目数量以降低误差方差（AERA et al., 2014）。

5. 计算与设计题（Spearman–Brown 预言公式） 题干：某量表含 10 个条目，现有内部一致性为 0.70。若维持条目质量与同质性，希望将信度提升至 0.85，需将条目数增加为原来的多少倍？据此估算所需条目总数，并说明该方法的适用前提与局限。 参考答案要点：

• Spearman–Brown：k = [ρ_target(1 − ρ_current)] / [ρ_current(1 − ρ_target)]
• 代入：k = [0.85 × (1 − 0.70)] / [0.70 × (1 − 0.85)] = 0.255 / 0.105 ≈ 2.43。
• 需约 2.43 倍条目，即约 24–25 题（在 10 题基础上新增约 14–15 题）。
• 前提与局限：要求新增条目与原条目平行/同质；仅增加数量可能引入冗余与疲劳，不解决维度性或质量问题；应并行关注条目质量、维度结构与内容覆盖（Nunnally & Bernstein, 1994；DeVellis, 2017）。

参考文献（APA 第7版）

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. AERA.
• Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220.
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
• DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE.
• Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412.
• Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley.
• McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum.
• McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
• Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154.
• Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. SAGE.
• Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.
• Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120.

Below are five evidence-based self-assessment questions designed to target core competencies in business email writing. Use a 1–5 scale (1 = not at all; 5 = consistently and effectively) to rate your draft.

1. Purpose and audience focus: To what extent does the opening state the main point or request clearly and early (BLUF) while framing relevance to the recipient’s needs, constraints, and decision context? (Department of the Air Force, 2015; Plain Language Action and Information Network [PLAIN], 2011; Garner, 2013)

2. Subject-line specificity: To what extent does the subject line concisely and accurately preview the message (e.g., action needed, topic, deadline), enabling efficient triage and retrieval? (Garner, 2013; PLAIN, 2011)

3. Organization and readability: To what extent is information front-loaded and segmented for scanning (short paragraphs, informative headings, bullets where appropriate), with concise sentences that reduce cognitive load? (PLAIN, 2011; Department of the Air Force, 2015)

4. Tone and professional etiquette: To what extent does the email maintain a courteous, professional tone appropriate to the relationship and context (clear but not abrupt; avoids slang, emojis, and ALL CAPS; uses bias-free and positive language)? (Garner, 2013; PLAIN, 2011)

5. Actionability, completeness, and correctness: To what extent does the email specify a clear next step, responsible party, and deadline; reference and correctly attach needed materials; use CC/BCC intentionally; present a professional signature block; and exhibit mechanical accuracy (grammar, punctuation, and formatting)? (Garner, 2013; Department of the Air Force, 2015)

References

• Department of the Air Force. (2015). The Tongue and Quill (AFH 33-337).
• Garner, B. A. (2013). HBR guide to better business writing. Harvard Business Review Press.
• Plain Language Action and Information Network. (2011). Federal plain language guidelines. https://www.plainlanguage.gov/guidelines/