心理學專家與開放科學專家對預註冊遵循度辨識的差異性

Abstract

近年來，科學界發現許多已發表於期刊的研究結果難以被他人重現，引發對科學知識長遠發展的擔憂。為提升研究通透度，預註冊（preregistration）逐漸被視為用以區別探索性與驗證性研究的實踐方法。然而，愈來愈多的研究表明預註冊計畫與最終發表的論文之間，往往存在未被揭露的不一致之處，可能影響研究的可信度，且期刊通常缺乏足夠的人力或資源進行預註冊遵循度審查。在開放科學概念的推廣下，學術圖書館憑藉其成熟的系統與豐富的資源，也逐漸成為開放科學服務的提供者，值得探索這類非特定學科領域的開放科學專家，在預註冊遵循度評估中所能協助的潛在角色。 對此，本研究以心理學為例，探討開放科學專家在協助評估預註冊偏差的潛力。在第一部分透過內容分析法，依循了一群心理學專家評估預註冊偏差的框架，對25份預註冊計畫和文章進行分析，比較開放科學專家與心理學專家對預註冊偏差的評估差異。第二部分則採用焦點團體法，討論開放科學專家與心理學專家對預註冊審查協議的形式偏好，探索可如何提供審查者更有效的支持。 研究結果顯示，開放科學專家與心理學專家對相同的預註冊計畫樣本的遵循度評估，一致率為72.6%。兩者評估的預註冊審查項目中，結果最為分歧的為研究的排除標準，且開放科學專家在不熟悉心理學的隱性規範下，可能難以辨識領域特定的知識，進而將描述較為模糊的內容視為非偏差。此外，關於對預註冊審查協議的形式偏好，開放科學專家較為傾向有提供具體操作指示的審查協議，心理學專家則更重視審查過程中的自主性。不過整體上，兩群專家皆認為理想的預註冊審查協議，應結合結構式與開放式的回應，以便審查者能清晰地闡釋其審查判定理由。 針對上述研究發現，本研究歸納出可通用於不同專業背景的審查者之預註冊審查協議設計原則，共涵蓋三個面向，第一，「賦能」審查者，使其具備足夠知識與能力進行審查；第二，「促進效率與效能」，透過降低審查者的認知負荷來優化審查過程；第三，「深化互動」，確保審查者的審查意見能充分回饋至被審稿者。藉由整理出預註冊審查協議的設計原則，有助於未來期刊重視預註冊遵循度的議題時，能有效地支持更多人參與審查以及完成審查任務。

As scientific fields discover that many published research results are difficult to replicate, concerns about the long-term development of scientific knowledge have arisen. To enhance research transparency, preregistration has gradually been recognized as a practice to distinguish between exploratory and confirmatory research. However, increasing evidence shows that there are often undisclosed deviations between preregistration plans and the final published papers, which can undermine research credibility. Furthermore, journals typically lack sufficient resources to review preregistration adherence. With the promotion of open science, academic libraries, utilizing their well-established systems and abundant resources, are becoming service providers for open science. It is worth exploring the potential role that these open science experts, not limited to specific disciplines, could play in evaluating preregistration adherence. This study uses psychology as an example to investigate the potential of open science experts in assisting with the evaluation of preregistration deviations. The first part employed content analysis, following a framework used by psychology experts to assess preregistration deviations. Twenty-five preregistration plans and articles were analyzed, comparing the differences in preregistration deviations assessments between open science experts and psychology experts. The second part used focus groups to discuss the preferences of both groups regarding the format of preregistration review protocols, exploring how to provide more effective support to reviewers. Results show that the agreement rate between open science experts and psychology experts on adherence assessments of the same preregistration samples was 72.6%. The most significant discrepancies in assessments occurred in the studies’ exclusion criteria. Open science experts, unfamiliar with implicit norms in psychology, may struggle to recognize domain-specific knowledge, potentially interpreting ambiguous content as no deviations. Regarding preferences for the format of preregistration review protocols, open science experts favored protocols that offered specific operational instructions, while psychology experts valued autonomy in the review process. However, both groups agreed that an ideal protocol should combine structured and open-ended responses, allowing reviewers to clearly articulate their reasoning for review decisions. Based on these findings, this study outlines design principles for preregistration review protocols that can be applied across different professional backgrounds. These principles cover three aspects: First, “empowering” reviewers by ensuring they have sufficient knowledge and capability to conduct reviews. Second, “enhancing efficiency and effectiveness” by reducing reviewers’ cognitive load during the review process. Third, “deepening interaction” to ensure that reviewers’ comments are fully communicated to authors. By establishing these design principles for preregistration review protocols, the study aims to effectively support broader participation in preregistration reviews as journals prioritize preregistration adherence issues in the future.