同儕審查的評審標準、信度與公平性研究：以台灣出版之社會暨人文科學期刊為例

Abstract

同儕審查是學術界進行科學探索時所採用的一項自律機制，幾乎已制度化地納入學術組織的運作之中，並普遍獲得學界人士的支持。基本上同儕審查的正當性是基於學術社群成員之間的信賴與誠信，在各項學術活動中以不同的作業模式分配有限資源，包括學術文獻出版、研究計畫獎助、大學教職聘用與升遷，以及學術成就獎勵等。但是同儕審查的運作方式迄今未臻完善，除了出現效用、效率，以及信度等問題外，許多研究亦已證實存在多種評審偏見，包括機構偏見、交情偏見、年齡偏見，以及保守偏見等。 本研究之研究目的有三，其一為回顧同儕審查機制之起源與發展，並分析同儕審查做為知識研究主體的歷程，綜整結論如下：同儕審查評審機制逐漸開放與透明、同儕審查的國際交流持續進行、同儕審查評審品質與效能提升的作法多元，以及同儕審查評審作業的未來發展，包括建立持續監督、檢驗與改進的同儕審查機制及同儕審查與書目計量的競合關係。其二是探討同儕審查之評審標準、信度與公平性的研究現況，綜整結論包括：評審標準的研究依然受到重視、評審信度過低的因果研究有待加強、公平性研究的方法論受到質疑，以及同儕審查的效用仍待證實。 另外本研究亦以3種我國出版之社會暨人文科學期刊之評審報告進行實徵研究，探討評審者在審查過程中使用的實際標準，並檢測評審者彼此之間的信度，以及討論評審的公平性與課責性議題，研究發現：評審報告的內容分析呈現評審者實際使用之核心標準、出版建議分析可細部解讀拒絕稿件或接受稿件之主要理由、稿件研究類型及稿件領域分析呈現非量化研究及人文科學研究的評審特性、公平性議題需審慎推論因果、評審評語與出版建議的一致性應進行系統討論、評審者行為理論之研究有其必要性，以及我國人文科學研究有其特殊的撰稿方式。 綜合而言，學術領域越來越專精且複雜，研究人口也愈來愈多，在學術資源未見大幅成長的背景下，競爭將日趨激烈，同儕審查研究的益受重視將不言可喻。證諸許多著名同儕審查期刊的稿件接受率只有個位數字，許多國家的政府奬助機構的獲奬率也有逐年降低的趨勢，某些頗具聲望的奬學金甚至僅有1/200的機率；另外各國高教經費緊縮及學生人數減少，也助長了大學教職之路的競爭。因此未來同儕審查的作業勢必面對外界更多的質疑與挑戰，而建立一個持續性監督與檢驗的機制是學術界的努力方向，進而營造一個開放的同儕審查程序、促進學術資源分配效率，以及健全學術研究環境。

Peer review is a self-regulation mechanism for scientific inquiry. Institutionalized and incorporated into the structure and operation of science, it has received considerable support in the academic setting. The legitimacy of peer review is based on trust and integrity. In various ways, it allocates scarce resources such as journal space, research funding, faculty recruitment, career advancement, and rewards for academic achievements. But there are growing indications of unresolved deficiencies in the operation of peer review, leading to negative assessments as to whether it is effective, efficient, or reliable. Many studies have found links between potential sources of bias and judgments in peer review, such as institutional prejudice, cronyism, ageism, and conservatism. This study aims to achieve three objectives. First, it explores the origins of peer review and traces the process by which it has become a subject of academic research. This examination shows that peer review has gradually become more open and transparent, has inspired ongoing international exchange, and has embraced diverse approaches for higher-quality evaluation. There is also increased anticipation for mechanisms to be established for continual supervision, scrutiny, and improvement, as well as for a competitive–cooperative relationship to develop between peer review and bibliometrics. Second, this study seeks deeper insights into contemporary research on different assessment criteria and on the reliability and fairness of peer review. While review criteria continue to be a major focus of research, there is also a need for greater investigation of the reasons for low inter-evaluator reliability, doubt has been cast on the methodology of studies on the fairness of peer review, and the effectiveness of peer review remains to be demonstrated. Third, this paper presents an empirical study of peer review reports provided by three social science and humanities journals published in Taiwan. For this purpose, the actual criteria employed are examined and the degree of reliability between evaluators is scrutinized; this is accompanied by a discussion on the issues of fairness and accountability in peer review. The findings include: the core criteria employed by evaluators, as revealed by content analysis of review reports; the main reasons for manuscript acceptance or rejection, as revealed by detailed interpretation of publishing recommendations; and the special attributes of peer review in non-quantitative and humanities research, as revealed by analysis of manuscripts in different research categories and specialized domains. These findings also highlight the need for caution in inferring cause and effect with regard to issues of fairness; for systematic debate with regard to consistency between evaluators’ review comments and their publishing recommendations; for research into the behavior of evaluators; and for consideration of the specific characteristics of manuscript preparation in humanities fields in Taiwan. Overall, academic disciplines are becoming increasingly specialized and complex, and the research population is growing ever larger, yet there has been no great increase in academic resources. In these circumstances, competition will grow increasingly intense, and thus it seems likely that research into peer review will attract ever greater attention. This growing competition is evidenced by the facts that acceptance rates for manuscripts submitted to many leading peer-reviewed journals are in single figures, and that rates of funding allocation by government funding organizations in various countries are also trending downward year by year, falling to a ratio of 1:200 at some renowned institutions. Budgetary reductions and declining numbers of students in higher education have also heightened competition among faculty. Therefore, peer review practice will inevitably face increasing questions and challenges from the outside world. Creating an environment for transparent peer review and establishing a mechanism for continual supervision and scrutiny should be a direction for concerted effort within academia.