Reliablity & Validity

come on let's go on a journey and learn about reliability and validity in research da da da welcome to another journey with Chris where we will obtain new levels of knowledge and awesomeness hello scholars today we're going to journey through reliability and validity on our journey we're going to cover types of reliability and types of validity reliability is the degree to which a research instrument an instrument is another word for a survey so it's the degree to which a research instrument produces consistent results have you ever seen the guy at the carnival who tries to guess your weight if he's right he gets to keep your dollar but if he's wrong you get a stuffed animal okay well let's say one day I go to that carnival guy and he guesses that I weigh 110 pounds okay let's say I come to him the next week and he guesses that I weigh 130 pounds well his guess isn't very consistent or reliable is it okay let's compare the carnival guy to my bathroom scale I step on the scale and I weigh 120 pounds a couple of days later I step on the scale and I weigh 120 pounds again now that's more consistent compared to the carnival guy trying to guess my weight it's not possible to exactly calculate the reliability or consistency of a research instrument but we can estimate it in a few different ways the first is test-retest reliability this is when the same instrument so the same survey is given to a group of people at two different times the results from those people from the first time they filled out the instrument are compared to the results from the second time in order to determine how well the instrument consistently produce the same results from those people the second is called parallel forms reliability this is when a group of people complete two similar versions of an instrument each version of the instrument are trying to measure the same thing then the results from the two versions are compared in order to determine the consistency of the results between similar versions of an instrument the third is internal consistent see reliability this is when different instrument items an item is another word for a question on the survey so this is when different instrument items that are trying to measure the same construct are compared see how they produce similar results there are two types of internal consistency reliability the first is average inter item correlation this is when a group of people complete an instrument and then all of the items on the instrument that are measuring the same construct are compared to each other one at a time in pairs then the items are compared overall to occur to create an average of those comparisons ok the next is split half reliability this is when a group of people complete an instrument and then all of the items on the instrument that are measuring the same construct are split in half so it's split into two groups to form two sets or two groups of items then the two sets of items are compared to each other to see how well they consistently measure the construct that's split half reliability okay the next is inter-rater reliability this is when the decisions from different raters a rater is a researcher who rates or judges a variable or variables in the research study so this is when the decisions from different raters are compared to each other to see how consistent the different judges decisions or ratings are for instance have you ever been to the website ratemyprofessor.com sometimes it seems like a professor is rated the same way by all of the students sometimes it's the other way around sometimes half of the students give a positive rating and the other half give a negative rating about that professor inter-rater reliability compares all of the ratings from each rater or judge to see how consistent they are if a professor on ratemyprofessor.com always gets positive ratings then there is good interrater reliability among the judges or the raters in this case it's the students if half of the students hate the professor and the other half love the professor then there is low inter-rater reliability among the students okay let's talk about validity validity is how accurate an instrument is at measuring what it's trying to measure let's go back to the weight example let's say I hop on my bathroom scale every day and every day it says that I weigh 120 pounds that's reliable that's consistent but let's say that the scale is off by 50 pounds because I actually weigh 170 pounds so even though the scale is reliable it's not valid it's not accurately measuring how much I really weigh there's a bunch of different ways of determining the validity of a research instrument the first is face validity this is when researchers simply look at the items on the research instrument and they give their opinion if the items appear to accurately measure what they're trying to measure face validity is the least scientific way of determining if your instrument items are valid face validity is just the researchers opinion if items look valid or not the second is construct validity this is when we are able to generalize about our construct of interest because we are actually measuring that construct accurately measuring that construct think of construct validity as being truthful and how we are labeling our construct for example let's say that we are measuring a construct called the love of coffee like what we did in class but all of our instrument items are about enjoying the smell of coffee so are we really measuring the love of coffee can we really talk about the love of coffee since our instrument only has items about enjoying the smell of coffee the answer is no we can only generalize or talk about in our findings the smell of coffee if we try to generalize about the love of coffee even though we're only asking about people's enjoyment of the smell of coffee we would have low construct validity ok let's talk about criterion-related validity this is when the results from the instrument accurately relates or predicts some kind of external variable that's related to the constructor that the concept or whatever it is that we're studying the classic example is SAT scores and it's criterion its its external variable of college performance so here look the SATs are designed to measure literal literacy skills writing skills and how well test takers analyze and solve problems the SATs have criterion-related validity by comparing SAT scores to an external variable that relates to SAT scores like college GPAs those who score well on the SATs tend to get good grades in college so a college GPA is the external variable that is used to show that the SATs have criterion-related validity finally let's talk about content validity this is how well an instrument covers the range of meanings included within a concept that's being measured for example if we were measuring the love of coffee we need to cover the wide range of meanings and indicators regarding the love of coffee so we will include items not just about enjoying the smell of coffee but also enjoying the taste of coffee or the effects of coffee or and so on or if we're trying to measure tobacco use we don't just measure measure cigarette use we have to measure all forms of tobacco use so cigarettes smokeless tobacco hookah um every snuff dip everything that we're looking at with tobacco use in conclusion reliability is the degree to which a research instrument produces consistent results validity is how accurate an instrument is at measuring what it is trying to measure there are several different types of reliability and validity thanks for a great journey and I'll see you in class

Channel: ChrisFlipp

Views: 496,468

Rating: 4.7984586 out of 5

Keywords: research, methods, reliability, validity

Id: 9ltvDNAsO-I

Length: 8min 19sec (499 seconds)

Published: Mon Mar 17 2014