麻醉學專科筆試項目能鑑度反應分析

Abstract

專科醫師測驗的目的在於評估考生是否足以勝任執業的最低要求。雖然專科醫師測驗對醫療照護品質而言相當重要，但是目前仍缺乏相關研究對專科醫師測驗結果進行詳盡的試題分析。事實上測驗中的項目反應隱含豐富且有價值的測驗訊息值得進行更多的相關研究。有鑑於此，本研究的主要目的為利用項目反應模式針對2007至2010年的台灣麻醉專科醫師筆試測驗進行廣泛的項目反應分析。 這四個年度的麻醉專科醫師筆試測驗均為100道單選題，應考人數介於34至37人之間。本研究採用兩種不同分析策略進行項目反應分析，先利用最大概似估計法估計模式參數與測驗信度，再將貝氏項目反應分析應用在更複雜模式的參數估計、不同模式的模式比較、評估共變數對考生能力的影響與多階層項目反應分析。 研究結果顯示這四個年度台灣麻醉專科醫師筆試測驗的信度介於0.71至0.75之間。兩種估計方式都可以得到單參數項目反應模式的考生能力與試題難度參數。但是在估計更複雜的雙參數與三參數模式時，最大概似估計法會遭遇無法收斂的問題。而貝氏法所得到的三參數模式估計結果顯示有過度參數化的疑慮，因此將所有猜題參數設為相等重新進行分析後發現這個共同參數的值接近於0。模式比較結果有利於採用單參數項目反應模式。而所收集到諸如考生年齡、性別與其訓練中心地理位置等變項對考生能力皆無顯著影響，階層項目反應分析結果顯示來自於同一中心考生彼此間的能力有相關性存在。 本研究證實了針對台灣麻醉專科醫師筆試測驗所進行的項目反應分析可以為將來的命題提供有用的資訊，而貝氏項目反應分析的彈性與多功能性對台灣麻醉專科醫師測驗的試題分析具有重大價值。

Board certification examinations for medical specialists aim to evaluate whether an examinee is competent to exceed minimum requirement for clinical practice. Although board certification examinations are of paramount importance to the quality of medical care, there is still lack of thorough investigations which focused on item response analyses of board certification examinations in a medical specialty. Item responses in a test are influenced by the examinee ability and item difficulty which require an in-depth statistical analysis. Therefore, the major goal of this thesis was to conduct comprehensive item response analyses on written tests of the Taiwanese board certification examinations in anesthesiology from 2007 to 2010 using a series of item response theory models. Data were derived from one hundred multiple choice items with single best answer included in each certification examination. The number of examinees ranged from 34 to 37 in each year for these four years. Two analytical strategies were applied to the item response analyses on the written tests of the Taiwanese board certification examinations in anesthesiology. The maximum likelihood estimation (MLE) method was used at first to estimate the parameters of the examinee ability and item difficulty and evaluate test reliability based on the one-parameter logistic (1-PL) model, so-called the Rasch model. Bayesian item response analyses were applied to dealing with more complicated item response models, including the two-parameter logistic (2-PL, considering item discrimination) and three-parameter logistic models (3-PL, considering guessing parameter). Bayesian approach was also used to assess the effects of covariate such as age gender, and geographic area on examinee ability. Bayesian multi-level model was also adopted to consider hierarchical data resulting from the correlation of item response within the same training center. The test reliability of written tests of board certification examination in Taiwan ranged between 0.71 and 0.75 in these four years. Both analytical approaches could estimate parameters of examinee ability and item difficulty in the one-parameter logistic item response model but the MLE methods encountered convergence problems during parameter estimation of the 2-PL and 3-PL item response models. The 3-PL model without restriction on guessing parameters based on Bayesian methods may lead to overparameterization. The common guessing parameters in the restricted 3-PL models with Bayesian approach were close to 0 in all the certification examination in anesthesiology held during the four-year study period. Model comparisons based on deviance information criteria provided evidence in favor of the 1-PL model. The effects of examinee characteristics such as gender, age and location of training centers on ability levels of examinees were not statistically significant. The application of multi-level Bayesian model to hierarchical data revealed correlation between ability levels of examinees from the same training centers. The effect of training center on examinee ability was not salient. This thesis demonstrates that item response analyses on written tests of the Taiwanese board certification examinations can provide useful information on test development in the future. The flexibility and versatility of Bayesian item response analyses were of great value for test analysis on written tests of the Taiwanese board certification examinations in anesthesiology.