COMPUTER – BASED TESTING
Computer based testing is often called Multiple Choice Questions (MCQ) but there are more possibilities to CBT than just that. Others call it the monkey-puzzle, but it only becomes that when designed by Great Apes. CBT can be made challenging and is a good method of assessment. It requires a lot of thought and analysis in its preparation, but it is a joy for the assessing team because marking is done flawlessly, instantly and automatically. Still better, the computer gives feedback on each question and enables the examiner to see its degree of difficulty, possible flaws in its wording, and points to where it can be improved. Apart from MCQs, there are Matching Item Questions, Area Specific Identifications, True/ False questions and other forms of CBT but these are not dealt with in this guide for setting questions, which is limited to the MCQ form.
Remember always that the purpose of the examination is not to trip up or catch out candidates, but rather to establish if they have the required minimum skills, knowledge and understanding expected of newly qualified graduate. Specialised abilities are not expected and must be avoided in all questions. Try to set question based on real practice and frequently encountered or major problems that a graduate must be able to deal with.
All questions will be subjected to review and improvement, but veterinarians can help the SAVC by supplying a variety of proposals based on their own experiences and requirements.
The MCQ consists of (1) a statement, question or problem. This is called the stem. Ideally it should be possible to give the answer, based only on the stem. (2) a series of possible answers called options from which the candidate selects one or more correct answers, depending on the marks allocated. The wrong answers are known as distractors. Most MCQs have only 1 correct answer and 4 distractors, to lower the risk of guessing or giving pure chance a high probability of affecting the outcome.
This should consist of relatively short, single question, statement or problem. Information or options in the stem must not be repeated in the answer options. If the stem takes the form of a negative statement, question or problem this must be made very clear by using bold capital letters in words like NOT, EXCEPT, WORST.
There must be only one obviously correct answer, unless the marks and questions clearly state that more than one answer is required. The distractors must be clearly wrong but seem plausible to candidates with insufficient knowledge or understanding. Try to make the answers similar in the sense that one does not stand out as clearly different. This applies also to the length of the option used. Avoid using complex answers like (a), (b) and (c) ; (a) and (d); all the above; none of the above. For numerical answers give these options in ascending order.
Try to achieve the following:
• The question or statement or problem must relate to important aspects and expectations in knowledge or skills in the subject within veterinary science
• The stem must be clearly worded
• It must be unambiguous
• It should be confined to one fact, idea or concept
• The stem should contain most of the information and thus leave the option short and clear
• Use direct questions or statements that lead to a definite answer
• There must be only one option that is definitely correct or clearly the best answer (unless several marks are to be awarded)
• Other options must all be definitely wrong
• Nevertheless they should all be plausible alternatives for candidates with insufficient knowledge or abilities
• Make the option as similar as possible to avoid inadvertently giving away the likeliest answer
• Similarly make the option of equal length where possible
• The options should link directly to the stem, ie form complete sentences when read together
Try to avoid the following:
• Vague wording, leading to uncertainty
• Ambiguous wording, leading to misinterpretation
• Multiple, long winded statements or questions with unnecessary information, leading to confusion.
• Tricky questions based on strange or unique experience, or your own specialisation and expertise
• Negative questions linked to some negative options. Double negatives are double trouble
• Options that enable a crafty candidate to deduce the correct answer without knowing the facts
• Complex, multiple options like (a), (b) and (e); (c) and (f); all of the above: for just 1 mark
• Relative terms like usually, sometimes, maybe
• True/False questions, as the candidate has 50% chance of guessing correctly