Effect of surprise test and instruction for negative marking on item analysis in Pharmacology

Introduction: The effect of instructions of negative marking on item analysis during an MCQ examination was studied.
Material and Methods: It was an observational type of longitudinal study conducted in MBBS students of second year. The students were given the same MCQ test twice with surprise. There were no instructions regarding negative marking during the first test (T1), whereas instructions were given for the second test (T2). The Raw score, Negative score and Corrected score along with Difficulty and Discrimination index of each MCQ was calculated.
Results: There was no statistically significant between the indices when the values of T1 and T2 were compared. There was statistically significant inverse correlation found between the difficulty index and the number of students who did not attempt the respective MCQ (Spearman correlation coefficient= -0.7818).
Conclusion: The Difficulty Index and Discrimination Index do not change with Conventional test and test with instructions of negative marking. Thus, these indices form an important tool for MCQs irrespective of the conventional and negative marking. 

Keywords: Difficulty index, Discrimination index, Multiple choice questions, Negative marking, Surprise test.


Introduction
MCQs (Multiple Choice Questions) are the integral part of assessment of medical students in formative and summative assessment. A good quality MCQ assesses the knowledge and differentiates various abilities of a student. 1 MCQ consists of direction to students, a stem and alternatives. Usually an MCQ has multiple choices (generally four), of which one is correct while the remaining three are known as distracters. 2 Item analysis evaluates the reply of students to each MCQ and suggests its quality and scope of perfection. The indices, namely difficulty Discrimination indices are the important tools of item analysis. 3 In a study by Narwane et al, 4 it was observed that there was a negative impact on the performance of students, when the instructions of negative marking were given. Therefore it is also likely that the parameters of item analysis would also be affected. The literature on the effect of instructions of negative marking on the item analysis is scarce. Therefore, the present study therefore was planned to see the effect of instructions of negative marking on item analysis during an MCQ examination was studied.

Materials and Methods
It was an observational type of longitudinal study conducted in MBBS students of second year. The students were given the same 20 MCQ test twice with surprise. There were no instructions regarding negative marking during the first test (T1), whereas instructions were given for the second test (T2). The Raw score, Negative score and Corrected score along with Difficulty and Discrimination index of each MCQ was calculated. The students used their own alphanumerical code as described by Narwane et al. 4 All the coded papers were assessed by a teacher unaware of the identity of students and the purpose of this educational study. After the completion of assessment, difficulty and discrimination index during T1 and T2 for each MCQ were calculated using Raw score, Corrected score and Negative score. The negative score was calculated by the addition of one mark and subtraction of 1 mark for each right and wrong answer, respectively. Marks were neither subtracted not added for a non-attempted question. The corrected score was calculated by the following formula Where n= Number of options for MCQ. Marks were neither subtracted not added for a non-attempted question 3 . Formula for difficulty index (DI)

Formula for discrimination index (DCNI)
Here, the students were ranked in a descending manner as per their overall performance. The upper 1/3 rd and the lower 1/3 rd formed the High score group and Low score group, respectively. H (among High score group) and L (among Low score group) are the number of students who answered the respective MCQ correctly. N is the number of students in high and low score group.

Results
Sixty seven students participated in the study. On comparing the respective DI calculated for Raw, Negative and Corrected score of T1 with that of T2, no significant difference was found (Table 1). Similar results were seen with the DCNI (Table 1).

Discussion
Allocating marks to right answer (Raw score) is the most commonly favored methods of assessment of multiple choice questions 5,6 . To avoid guessing of answers, negative scoring is used, where negative marks are awarded for an attempted question with wrong answer 7 . It has been observed that there was increase in total score in students who guess answers. [9][10][11] Guessing also lowers the validity and reliability by introducing a random factor into test scores. 12,13 Test designers cannot distinguish between correct answers attributable to knowledge mastery as compared to those based on a guess. 14 However, it also discourages students from answering questions where they are less confident. The corrected score method takes into consideration the number of options provided in the MCQs. After excluding the right alternative, the student has probability of being wrong becomes 1/(n-1), for instance, 33% for an MCQ with four alternatives. Therefore, the marks subtracted for each wrong answer is 1/3 instead of 1, for opting one of wrong alternatives.
As shown in Table no. 3, on comparing the Difficulty indices for Raw, Negative and Corrected score of T1 with that of T2, there was no statistical significance found (Paired t test). Similar results were seen on with Discrimination index (Paired t test). Hence these indices do not change irrespective of the system of marking as well as instruction for negative marking that were used for evaluation of the MCQs. There were no studies found in literature that studied and published regarding the differences in scoring system with respect to surprise test.
The Difficulty indices of each MCQ were correlated with the Number of times it was not attempted (Table 4, Fig. 1). There was highly significant negative correlation (P<0.0001, Pearsons Correlation test, r = -0.8154) between the number of times the MCQs were not attempted in the T1 and T2. Thus, the probability of not attempting a question increases with increase in the level of difficulty of the question.

Conclusion
The Difficulty and Discrimination indices are valid parameters irrespective of the type of scoring system used for MCQs. The number of non attempted questions increase as the difficulty index decreases for a given multiple choice question.
Source of funding None.