Mar 30, 2015: Measurement Matters: Personality Inventories from a Participant Point-of-View?

A couple of weeks ago, I sent my personality students off with a personality inventory and told them to bring it back to class, where we would score it. The inventory was the 60-item version of the English self-report HEXACO-PI (Ashton & Lee, 2009).

I asked the students what traits they thought were being measured, and whether they thought a friend or family member would describe them the way that they described themselves. I also asked them what traits the questionnaire was missing. That is, what were the important aspects of their personalities that simply weren’t captured?

I have a terrific group of students, and their answers were insightful. Their descriptions of what was being measured by the inventory were generally in keeping with the six HEXACO personality factors: Honesty-Humility, Emotionality, Extraversion, Agreeableness, Conscientiousness, and Openness to Experience.

Student observations

Many students pointed out that it’s pretty easy for a friend or family member to accurately gauge their level of extraversion, because extraverted behavior tends to be easily observed. On the other hand, perhaps Mom doesn’t really know that they might use counterfeit money if they were certain of getting away with it (an Honesty-Humility item), and maybe their friends aren’t aware just how much they worry about the little things (Emotionality).

When it came to what was missing, the most common answer referred to religiosity or spirituality. In some research that has included religiosity/spirituality items, these items have indeed yielded a factor, suggesting that perhaps we should have a seven factor model of personality. However, Ashton et al. (2004) took the stance that religiosity and spirituality do not fall into the domain of personality proper and thus, these items do not appear in the HEXACO-PI. Other students suggested that their indecisiveness was not captured, and another suggested that her competitiveness was not adequately measured.

I was impressed at the thought my students put into their responses, but I was not particularly surprised. That changed when I read their responses to my request that they identify some strengths and weaknesses of using self-report inventories, such as the HEXACO-PI.

Some students honed in on exactly what I was looking for: they discussed the efficiency of personality questionnaires, as well as the issues of social desirability and variance in levels of self-awareness. Nobody commented on response biases, such as tendencies to agree or disagree with questionnaire items, but I introduced the topic in discussion and also recommended that they all take my third-year Psychometrics course.

Here’s where things got surprising: several students went in an entirely different direction when they addressed the strengths and weaknesses of the inventory. They pointed out that a strength of the inventory was that it might give people very general information about their personality, including aspects of their personality that make them happy, and those they might want to change. A weakness, they said, was that the inventory didn’t give them ideas about how to change, nor did it suggest which aspects of their personality were psychologically healthy versus unhealthy.

Participant vs. Researcher Perspectives

When I encountered the first response of this nature, I was bemused. Why would they think I was asking about the strengths and weaknesses from the benefit of the participant? Obviously, I was referring to the researcher’s perspective! By the time I read several such responses, I started to experience a sense of discomfort: why shouldn’t they think of the benefits to the participant?

I am not oblivious to ethical considerations when I conduct my research. I spent two years on a Research Ethics Board of one institution, later serving as their Student Mentor, advising my graduate student peers on how to conduct their research in an ethical fashion. I have also served on a research experience departmental committee.

My guiding principle in conducting my research is that no one should leave my study in a worse state than they walked in the (usually metaphorical) door. Obviously, I do not want to harm my participants in any way. However, my students seemed to be suggesting they want more than to leave a study unharmed. If they are going to participate in research, at least some of them are looking for learning experiences and growth opportunities.

Questions, Questions, Questions

Fellow researchers: Do you look for ways to promote learning and growth for your participants? How? Do you give them feedback on their results? Do you find it challenging with online studies, when you generally do not interact with your participants? Do you debrief? If you use a debriefing letter, is it meaningful and full of useful suggestions? Or, does it simply thank the participant and provide a list of counseling services just in case your participants have experienced emotional turmoil?

Participants: How could your questionnaire experience be improved?

Clearly, at least some of my undergraduate students, who I think are pretty representative of our student samples, are expecting more from their questionnaire research experience than many of us are giving them. Can the questionnaire research experience be made more meaningful for participants in a way that still allows us to get our research done in a reasonably efficient fashion? Feel free to comment!