A more effective model for employee assessment

Cognisco use Multiple Response Questions (MRQs), but what are the differences and similarities between these questions and Multiple Choice Questions (MCQs).  They are similar in respect that they both comprise of a question stem with numerous responses.  The differences, however, are:

Multiple Choice Questions

New multi choice question


MCQs give the individual taking the assessment a number of options that are presented as possible answers to a question stem, only one of which is correct.  The employee is required to select the correct response to the question stem out of these options.  This type of question is open to bias.  If a person takes a guess at which answer is correct out of, say, four options, they could be correct, one out of four times.  The employee’s results for this type of assessment could give a false reflection of their understanding of the subject being assessed.  While Sutherland worked with Cognisco in gaining an understanding of decision-making and irrational behaviour, he also believed that Multiple Choice Questions tell us more about a person’s decision making patterns than it does about their actual knowledge of a subject.

Multiple Response Questions

MRQs, on the other hand, also use four or five responses to each question stem, but the employee taking the assessment is required to answer whether each of these responses is right or wrong.

New multi response question


This means that a question stem with five responses could be answered in thirty-two different ways, so you only have a one in thirty-two chance of getting a question 100% correct, which all but eliminates the “luck” element that is found to a larger extent in Multiple Choice assessments.

In Cognisco assessments, we include an option of “I don’t know” for each response to give the employee taking the assessment an opportunity to state that they literally don’t know the answer to that particular response.  This increases the number of possible ways to address the question to a potential total of 243 choices.

When delivering the questions to the employee the questions can be randomised to reduce employees colluding.  Each employee will therefore not only receive the questions in a different order but each of the responses in each question will be in a different order as well.

Did you notice the confidence bar?

Confidence based learning goes back to the 1930’s and is a culmination of commercial, educational and government research that made connections between learning, knowledge, retention, correctness and confidence and learning.  These connections have been developed over the past 70 years with evidence from various researchers linking knowledge and confidence.  Conclusions from one researcher[1] have stated that knowledge alone is necessary but not sufficient to create a specific behaviour.  However, the combination of knowledge and confidence leads to behavioural change, which allows people to take action.  People who are correct and confident are more productive in the actions they take.  However, the opposite is also true for people who are confident about ‘misinformation’, as they can act in a way that is persistently negative with potentially dangerous consequences.

When an individual is given the option about the correctness of an action, we also need to take into account their level of honesty.  Most assessments push people into a corner and potentially ‘force’ them to give an answer as to whether something is right or wrong.  By offering an additional choice of ‘I don’t know’, the individual is given an option to say they can honestly say they don’t know.  This highlights the potential for personal development, safe in the knowledge that this person is more than likely to ask more questions or ask for help.

Once an individual takes our assessment, we can identify their level of mastery of a subject.  Mastery is an individual who is confident that they have the correct knowledge and can confidently put that knowledge into practice.  Therefore a validated achievement is about being 100% correct and 100% confident about the content they are being asked.

It is not all about the score the individual has at the end of the assessment.  Once an individual has completed one of our assessments, we would identify the gaps in knowledge and offer a targeted intervention until they have mastered the content of the subject being assessed.  It is therefore about getting individuals to master the subject being assessed so they can become confident, productive individuals.


[1] Bruno, J., Pavel, M., & Strand, S., (1995). Identifying and addressing information deficits for undergraduate students in science. Journal of Educational Technology Systems


By Cognisco

Sign up to our mailing list:

Comments are closed.

  • Subscribe To Our Podcasts

  • Feedback