Check out some similar questions!

kingkarl · Sep 30, 2009, 06:08 PM

I am working on a student's dissertation and have come across a use of Likert survey questions to establish groups to analyze for differences. What the student has done is use a series of questions to break the respondents into groups... those with higher than average responses on a series of questions and those with lower than average responses. He has used the middle of the likert survey scale (in this case 1,2,3,4,5) with 3 being the average or middle. If a respondent has answered 1 or 2 he is segregated into a "lower" category. If a respondent has answered 3, 4 or 5 he is segregated into a higher category. By this means the researcher has built two groups (lets call them lower and higher) and then looked for differences in other question groups on the survey. I have never seen this done in social research before and wondered if this was an acceptable means to build groups other than using demographic data? Any and all feedback would be appreciated.

ballengerb1 · Sep 30, 2009, 06:28 PM

#3 is the median and the mean but he inlcuded that in the higher group, so that group is 50% larger than the small group, very curious grouping. Was he aligning the type of answer for 1 & 2 to a specific thread or type of answer? For example, were all the answers 1 & 2 Republican type answers while 3,4 &5 were Democrat answers.

Wondergirl · Sep 30, 2009, 06:50 PM

I don't agree with his use of 3. That should be neutral territory. He could even build a third group out of the 3's.

I wonder how reliable the data are, since respondents prefer rating scales with verbal cues. Numbered scales encourage central tendency bias (respondents avoid selecting answers at the extreme ends of the scales).

kingkarl · Sep 30, 2009, 08:18 PM

I don't disagree that the use of the median value biases the "upper group", over the "lower group", but was more looking for information as to whether Likert type answers can establish groupings to be difference tested. I can get him to change to a non Median likert survey (1,2,3,4). But, then what has he got and how does he test when part of the survey that is being tested for difference is used to differentiate groups.

Wondergirl · Sep 30, 2009, 08:52 PM

Originally Posted by kingkarl

I don't disagree that the use of the median value biases the "upper group", over the "lower group", but was more looking for information as to whether Likert type answers can establish groupings to be difference tested. I can get him to change to a non Median likert survey (1,2,3,4). But, then what has he got and how does he test when part of the survey that is being tested for difference is used to differentiate groups.

Does he replicate the Likert results by using another kind of test?

kingkarl · Oct 1, 2009, 06:42 AM

No, he doesn't, he simply extracts a portion of the likert survey and uses it to establish two groups and then uses the remainder of the survey to analyze for difference between the two contrived groups. Has anyone seen this done before??

Wondergirl · Oct 1, 2009, 09:16 AM

Originally Posted by kingkarl

No, he doesn't, he simply extracts a portion of the likert survey and uses it to establish two groups and then uses the remainder of the survey to analyze for difference between the two contrived groups. Has anyone seen this done before???

It seems like there should be something wrong with extrapolating data in this way, especially without other surveys that test the same information and may come up with other results. I really wonder if his results are valid.

kingkarl · Oct 1, 2009, 02:57 PM

I think I agree, I had never seen this done before and glad I checked... Would love to hear other opinions about or instances when this has been applied in other research.

ballengerb1 · Oct 1, 2009, 03:31 PM

I would say this is a faulty start for him unless all his questions were worded so the likert 1 & 2 would only come from one type of person like my example of a Republican and the Democrats would be more likely to anwer 3-5.

kingkarl · Oct 1, 2009, 07:12 PM

Its not that type of survey. Good observation though, I hadn't thought of that instance. This is a survey used in a Pretest, Postest, and Post Posttest after a seminar and I don't think there is any way to address polar opposits in this survey. Again, sharp observation, I think I will have him go back to looking at differentiating groups in his demographic section...

Thanks again

ballengerb1 · Oct 1, 2009, 07:23 PM

So the pretest was like establishing a baseline of existing knowledge, then knowledge gained from seminar and finally long term retention of that knowledge, along those lines? I can picture how this could be valuable to a presenter to determine his effectiveness but I need to stop here, Elvis's brain has left the store.

kingkarl · Oct 4, 2009, 02:23 PM

Exactly, this is classic pretest/post test and uses the paired t-Test to analyze for difference between the pretest knowledge and the posttest knowledge. This experimental design is classically used to evaluate a new teaching method, but is also used to evaluate diet programs (weigh, diet, weigh), blood pressure medicine effectiveness (blood pressure test, medicine for 6 months, then blood pressure testing again). A most effective, but slightly different way to analyze for difference in a common variable before and after a treatment.