Exam Development and Scoring

NCCPA’s exam questions are developed by committees comprising PAs and physicians selected based on both their item writing skills, experience and demographic characteristics (i.e., practice specialty, geographic region, practice setting, etc.). The test committee members each independently write a certain number of test questions or items, and then,each item then goes through an intense review by content experts and medical editors from which only some items emerge for pre-testing. Every NCCPA exam includes both scored and pre-test items, and examinees have no way of distinguishing between the two. This allows NCCPA to collect important statistics about how the pre-test items perform on the exam, which informs the final decision about whether a particular question meets the standards for inclusion as a scored item on future PANCE or PANRE exams.

When NCCPA exams are scored, candidates are initially awarded 1 point for every correct answer and 0 points for incorrect answers to produce a raw score. After examinees’ raw scores have been computed by two independent computer systems to ensure accuracy, the scored response records for PANCE and PANRE examinees are entered into a maximum likelihood estimation procedure, a sophisticated, mathematically-based procedure that uses the difficulties of all the scored items in the form taken by an individual examinee as well as the number of correct responses to calculate that examinee’s proficiency measure. This calculation is based on the Rasch model and equates the scores, compensating for minor differences in difficulty across different versions of the exam. Thus, in the end, all proficiency measures are calculated as if everyone took the same exam.

Finally, the proficiency measure is converted to a scaled score so that results can be compared over time and among different groups of examinees. The scale is based on the performance of a reference group (some particular group of examinees who took the exam in the past) whose scores were scaled so that the average proficiency measure was assigned a scaled score of 500 and the standard deviation was established at 100. The minimum reported score is 200, and the maximum reported score is 800.

We do not publish the percent correct level necessary to pass our examinations any more. Given that we have multiple test forms this information would not be accurate since some test forms, while built to be exactly the same, are slightly different in their difficulty. Therefore we convert the percent correct to a scaled score and report scores and the passing standard on that scale.