探討考官制度對於考試之影響因子：以中華民國齒顎矯正專科醫師測驗為例

Abstract

簡介 中華民國齒顎矯正專科醫師測驗由三部分組成：第一部分為筆試測驗，由100題單選式選擇題組成，分數需達及格標準始得參與下一階段測驗；第二部分需遞交符合中華民國齒顎矯正專科學會所公告之病例分類及資料等進行書面審查，所有送審資料皆須符合及格標準始得參與下一階段測驗；第三部分為口試測驗，前半部為自選病例測驗，於考生所送審之自選病例中隨機挑選一例作為自選病例測驗之測驗項目、後半部測驗為指定病例測驗，於學會所準備之題庫中隨機挑選一例作為指定病例測驗之測驗項目。前半及後半之口試分數經加權處理後達及格標準者始為中華民國齒顎矯正學會認可之專科醫師。 評估或是測驗會受到許多主觀因素之影響而產生偏誤 (bias)。同時偏誤也可能是由客觀因素所造成：例如考官之年齡、性別、種族、教育程度或是專業經歷等。考官及考生間交互作用之模型非常複雜。 截至今日於牙科領域中仍無相關研究探討考官偏誤對於測驗之影響。因此本研究以中華民國齒顎矯正專科考試為例，試圖探討測驗中產生考官偏誤之可能客觀因子。 材料與方法 本研究採用回溯式研究設計 (retrospective)，共收集2009至2019年間共50位考官資料及900筆測驗分數數據。考官共由29位男性、21位女性組成，且依照下述特性分組分析其敘述性統計資料及進行統計分析：(1)年齡 (2)性別 (3)專科經驗（年） (4)參與評審次數（次） (5)執業地區 (6)執業院所。 統計方式則將考生分數設定為依變項、上述考官客觀資料為自變項，於兩測驗分別進行簡單線性回歸 (univariate linear regression)，若於簡單線性回歸中發現該變項具有統計上顯著差異則會將該變項取出，綜合後進行多元線性回歸(multivariate linear regression)。拒絕需無假設之p值訂為0.05。 實驗結果 敘述性統計資料顯示：考官之平均年齡為48.32歲（標準差7.44）。取得專科年資項目中，取得專科年資之平均時間為13.06年（標準差為4.4年）。參與考官次數項目中，平均次數為2.7次（標準差為1.54次）。工作地區項目中，31名考官於北部地區執業、19名考官於中部、南部或東部區域執業。工作地點項目中，24名考官於教學醫院執業、26名考官於私人診所執業。 於自選病例測驗中，簡單線性回歸模型顯示下列變項之分數會具有統計上的顯著差異：年齡、取得專科認證年資大於15年之考官、工作地點為私人診所之考官。多元線性回歸模型中，下列變項之分數會具有統計上的顯著差異：年齡、工作地點為私人診所之考官。 於指定病例測驗中，簡單線性回歸模型顯示下列變項之分數會具有統計上的顯著差異：年齡、性別、取得專科認證年資大於15年之考官。多元線性回歸模型中，僅年齡變項之分數會具有統計上的顯著差異。 結論 考官偏差 (examiner bias) 確實會對測驗結果產生影響。以中華民國齒顎矯正專科醫師測驗為例，在其餘變項條件皆相同之狀況下，自選病例測驗中考官具有15年專科經驗者以及工作地點為私人診所者評分較高。於指定病例測驗中，考官具有15年專科經驗者評分較高，但女性考官分數較低。兩者考試中皆有考官分數隨年齡增長而增加之現象。多元回歸模型顯示自選病例測驗中較為年長之考官所給予之評分較高，此外在其餘變項條件皆相同之狀況下，於私人診所執業之考官相較於教學醫院執業之考官所給予之評分也會較高。於指定病例中評分則僅會受到年齡影響，較為年長之考官所給予之評分較高。

Introduction The examination of Taiwan Board of Orthodontist composed of three parts. The first part is written exam which composed of 100 multiple-choice. The second part are application documents (self-prepared cases). The final part is oral examination. Candidates have to select one case randomly from self-prepared case and the other one from assigned cases which was provided by board committee. The final scores should exceed 60 points from weighted scores of oral examination as qualified Taiwanese Board of Orthodontist. Assessment or rating is subjective to various form of bias. Examiner bias could be objective which means can’t be changed by examiner themselves like gender, age, race, educational level or professional experience. The interaction model between students and examiners is quite complex. However, no study about examiner bias was performed in dental education realm especially orthodontic field. This study aimed to figure out possible influencing factors in examiners from examples of Taiwan Board of Orthodontist examination. Material and methods The examiner groups composed of 29 male and 21 female who were selected from certificated Taiwan board of orthodontist. The score data was collected from 2009 to 2019. The examiners and corresponding scores were grouped as follow: (1) Age (2) Gender (3) Experience time (year) (4) Evaluation frequency (times) (5) Place area (6) Workplace The analytic method was linear regression models in univariate and multivariant ways. The models of self-prepared cases and assigned cases were conducted separately. The multivariate model would only include the statistic significantly variables in univariate regression model. The covariance was list above. The p value set as 0.05. Result The average age of examiners was 48.32 years old (standard deviation, SD= 7.44). There were 6 examiners under 40 years old, 24 examiners aging from 40 to 49 years old, 14 examiners aging from 50 to 59 years old, 6 examiners more than 60 years old. There were 13 examiners whose certification less than 10 years, 19 examiners ranging from 10 to 14 years, and 18 examiners experience period more than 15 years. The experience years was 13.06 years (SD= 4.40 years). The evaluation frequency among examiners was 2.7 times (SD= 1.54 times). There were 31 examiners practiced in northern area and 19 examiners practiced in another region. There were 24 examiners worked in hospital and 26 worked in private practice. In univariate linear regression models of self-prepared case, the scores were statistically significant in these following variables: age (p= 0.0003), experience time exceed 15 years (p= 0.0079), work at private practice (p= 0.0077. As for multivariate model of self-prepared cases, the scores were highly significant in groups of age (p= 0.0007) and examiners work at private practice. (p= 0.0190). In univariate linear regression model of assigned case, the scores were statistically significant in following variables: age (p<0.0001), female (p= 0.0209), and experience more than 15 years groups (p= 0.0007). As for multivariate model of assigned cases, the scores were statistically significant in age group (p= 0.0001). Conclusion Examiner bias may affect the rating result. As for exam result of self-prepared case from examples of Taiwan board of orthodontist examination, experienced examiner who get certification more than 15 years and the examiners who work in private practice tend to rate more lenient. As for assigned case in Taiwan board of orthodontist examination, the experience examiner who get certification more than 15 years tend to rate more lenient but female examiners tend to rate harsher. The scores would increase with examiner’s age increasing in both self-prepared cases and assigned cases. As for multivariate model of self-prepared exam, the elder and the examiner who work in private practice clinic tend to rate more lenient. As for multivariate model of assigned case exam, the score would be only affected by age. However, further studies are necessary to determine the direct relation among these factors.