You operationalize your variables in order to measure them.
So, now let's talk about measurement and related concepts.
When measuring your variables you may ask yourself...
Is my measure “on target”? That is, are my measurements accurate?
Do my measures “cluster together”? That is, am I getting consistent results?
But what does that mean?
What we are talking about is validity and reliability.
Let's start by thinking about how to measure prejudice in people. How would you do that? A survey? What would the questions be on the survey? Your measure of prejudice needs to be valid and reliable. Are you sure they are valid and reliable? Are you accurately measure the level of prejudice in a person with your survey? Does your survey get the same results with the same person each time?
Validity: “the extent that scales or questions do measure what they are thought to measure”(Stacks & Hocking).
You can think of validity using a target metaphor. Is it on target (i.e., near the bulls eye)?
Each "shot" on the target represents a measurement.
Or think of a bathroom scale. What does it meen to say a bathroom scale is valid or not?
Photo by elcamino73. Used under Creative Commons.
------------------
A related concept to validity is reliability.
Before looking at a formal definition of reliability, just think of the everyday use of that word. If you say your friend is reliable, what does that mean? It means you can count on your friend. Every time that you call on that friend they are there. Not sometimes. All the time. They are consistent. The formal definition of reliability is similar.
Reliability: “the extent to which measurement yields numbers (data) are consistent, stable, and dependable.” (Stacks & Hocking).
Again, let's use some metaphors to see the concept.
What about a bathroom scale and reliability? What does it mean to say that a bathroom scale is reliable?
Photo by elcamino73. Used under Creative Commons.
Can an instrument can be reliable, but not valid. That is, cluster together, but not be on target?
If you had a bathroom scale that was reliable, but not valid, what results would you get if you weighted yourself several times?
Example
Let's say we are interested in the topic of communication apprehension. More specifically, we are interested in the relationship between gender and communication apprehension. Do men or women have higher levels of communication apprehension? How would we go about answering that question?
How would we measure communication apprehension in our subjects (the people we are studying)? We could observe. What about a survey? Yeah, let's do a survey. Something like below.
-------------------------------------
Conversation Apprehension Scale
1. While participating in a conversation with a new acquaintance, I feel very nervous.
Strongly Agree --- Moderately Agree --- Neutral --- Moderately Disagree --- Strongly Disagree
2. I have no fear of speaking up in conversations.
Strongly Agree --- Moderately Agree --- Neutral --- Moderately Disagree --- Strongly Disagree
3. Ordinarily I am very tense and nervous in conversations.
Strongly Agree --- Moderately Agree --- Neutral --- Moderately Disagree --- Strongly Disagree
4. Ordinarily I am very calm and relaxed in conversations.
Strongly Agree --- Moderately Agree --- Neutral --- Moderately Disagree --- Strongly Disagree
------------------------------------
Think of this survey as a measuring instrument, just like a bathroom scale. The bathroom scale measures your weight and this survey would measure your communication apprehension.
Does our instrument (the above survey) have good measurement validity and measurement reliability? How would you determine that?
Measurement validity:
“the extent to which researchers are actually measuring the concepts they intend to measure”(FBFK)
Do the instruments give accurate/true readings?
Measurement reliability:
“the extent to which measurements of a variable are consistent and trustworthy”(FBFK)
Do the instruments continue to give the same readings every time they are used?
What are the procedures for checking an instrument’s reliability?
Similar results every time?
0% = Not reliable to 100% highly reliable
Three Ways to Check Instrument’s Reliability
1. Test and retest it.
2. Test, change wording slightly, retest.
3. Compare 1/2 items to the other 1/2
3 options, Not step-by-step
Which option is best? Costs and benefits?
--------------
Share this post with others. See the Twitter, Facebook and other buttons below.
Please follow, add, friend or subscribe to help support this blog.
See more about me at my web site WilliamHartPhD.com.
No comments:
Post a Comment
Thank you for your comment.
Your comment will be reviewed.
If acceptable, it will be posted after it is carefully reviewed. The review process may take a few minutes or maybe a day or two.