Focus Group Session 4 -Analysis

the fourth pillar that holds up the focus group is analysis and this is the one that kind of really depends on the other things the interesting thing about analysis is we've had people come to us and show us what they have and they were asking questions about analysis but when we looked at what they had how they captured the data the questions they had it was just about impossible to do analysis analysis kind of depends on how the other parts of it works in other words the questions did you ask the questions consistently or did you keep changing the questions one of the things we want to do is to have consistent questions or if you do change them just ever so slightly and under a fair amount of thought so you have the ability to compare one group to the other and so analysis depends upon how the moderator did their task did they follow up did they do all the things that were necessary to get the information there are two words that are critical when we talk about analysis and much of analysis sort of revolves around these two words and by the way this applies to both quantitative and qualitative analysis I mean in some ways there's some similarities here both of them are systemic that means there are a series of steps you follow I mean you've all had training and statistical kinds of things you know that you just don't take a bunch of numbers and you say oh what am I going to do with these today I'm going to be creative I'll do some things that no one else has done no you don't do that because what you want is it for credible to other people who are looking at it the same is true with qualitative analysis you follow the protocol that have been established of how you handle the how you capture the data how you document the data how you code the data all those things are necessary for you to be done in a systematic way and here's the interesting thing it's not important it's not only important for you to be systematic but you have to be able to explain it to others people will come to you and they'll say how'd you do analysis and what they're asking for is what was the protocol how did you do it how did you go from this conversation to this written report and what were the steps involved what were the decisions that were involved along the way and so you need to be able to articulate to explain what that is because what people are worried about is that you just picked the ideas that you liked right that's what they're worried about you know like did how did you do this like it's some kind of magic happen or did you just I pull this up so you have to be able to describe the process that you used also the second word is verifiable now that's sort of like the little bread crumbs you leave along the way the trail of evidence that you have and you you maintain and you keep all of those the reason is that if needed you should be able to replicate each of the steps along the way you haven't you have an audio recording by the way we really do like the audio recordings because they capture a great deal of information that's very hard to get when you're just trying to take notes you can get the the words that people use the actual words sometimes the quote is long but what we encourage especially we don't do every study this way but we often go from here to a transcript and then we'll basically do a content analysis of the transcript of looking for patterns that are occurring along the way but there's a series of steps someone could go back and verify each of those steps along the way and and part of this means there are some things that are done at different times like there's some things done while you're still in the focus group what are those barriers I'm just gonna back up for one one second and say that depending upon what kind of a study we're doing we don't always use the same process for analysis if you're doing it for your dissertation you're gonna have a different analysis process then if you're doing it for the neighborhood newsletter so time what we're looking at is you know how reversible are the idea the the decisions how big is the decision that's going to be made what are the standards for what it is that you're doing if you're doing it for a journal article or something there are different expectations so part of it is that we're looking at you know what for this particular study what's the appropriate level of analysis that we're gonna do and when we're doing analysis it's a little different in qualitative than quantitative and quantitative you often kind of wait until you have like lots of the day to end before you start doing analysis in quantitative in focus groups you start doing analysis while you're in the very first group and as the moderator what you're doing is you're thinking about am I getting enough information on this do I need them to be more concrete do I need an explanation so you're pushing for more data more information that's gonna be useful when you get to analysis in the literature they call this emergent and that's an important factor that it it's not so true in quantitative research but in qualitative research each time you do it you your understanding your knowledge about what's happening is increased and so this the process of analysis is emergent as you go along you know more and more insight about whatever it is you're looking at and that usually occurs to a certain point where you reach what we call saturation and and then the way that we capture the data in the focus group is that we'll always have a note taker and so they're in the whoever is doing the analysis should tell the note taker what kind of notes they want because sometimes if you just say take notes you'll get like back minutes to a meeting which is not helpful when you're trying to do analysis and so the whoever's doing the analysis should tell the note taker or maybe you want the analyst to be the note taker I mean I do that like in the minutes of a meeting you have alread minutes of a meeting rarely if ever do you see a quotation someone actually saying something however in a transcript of a focus group it's pretty much all quotations direct comments said by people that's the evidence you want to use when you're preparing the report the summary is interesting but it's the quotes that the evidence you use to support those larger points so we've got the note-taker we've got the audio recording you'll gather all the things that people produce so if they drew something if they made a list of something if you've written things on the whiteboard you'll take a photo of it so you're gathering all the information that's part of your data you might have somebody in the back be with a laptop if it can be quiet you might use somebody taking notes that way with a laptop and then in market research though to use video we don't recommend that for a lot of different reasons but you know what you want to do is think about kind of those lower levels that you've got you've got the notes you've got the audio recording you've got all the information that people wrote or whatever and that'll be your data stream and then we often also ask a final question while we're still in the groups that helps for them they also standpoint it's kind of that last question dick of all the things we talked about here today what's most important or if you had a minute to talk to the people who are designing this program what advice would you give them and you go around so that's kind of what you do in the group from an analysis standpoint and then right after the group then you're gonna do debriefing now your first instinct will be to go home because it takes a lot of energy to do these but resist that temptation and sit down together and what you'll do is first you'll check the audio recording to make sure that you have something if you don't you'll go question by question and try to take as much notes as you can but if it's if you've got something turn it back on and now we'll have a conversation about what we should pay attention to in this group so we'll think about what seemed to be the important ideas that came out this is called the debriefing yep so we're having this right after the focus group conversation and it's what what seemed to be important were there any quotes that seemed really kind of standout kind of quotes that we want to make sure and capture I'm gonna differ from other groups were there any surprises in this one is there anything that we need to change about recruiting anything that we need to change about the questioning route was there something that didn't work as well so all of that will record it might be a 15 20 minute conversation and then and then you can go home but you can actually use that then if somebody wants a summary of the focus group you can just go back and transcribe some of that debriefing and you'd have a pretty good kind of little summary of what went on what you're doing here is doing really a memory dump that your memory is really good immediately after that group 24 hours it's gonna fade so to another group it's gonna erase almost completely and so while your memory is still fresh people have left the group you sit down and you take it from your memory and you put it into another permanent form that you're not going to lose and also that's the first time that the members of the team are starting to talk with each other and they might in that conversation that could even evolve into a higher level where they're talking about something they're seeing something they're observing something taking place but that debriefing is a valuable part of the analytical process that you're going through and then after that well then you've so now the group is over and you've got a bunch of material you've got field notes you got audio recording you've got a debriefing that's having afterwards and then now depending on the study sometimes this decision is made earlier it's a question of should you transcribe it or not you know if you're doing a dissertation you're doing a major kind of a study where you're trying to get it published it's a really important research study the answer is almost always yes you probably do need to transcribe but if you're doing it for some community organization where speed is really important they need results within two days and or the resources are really limited they don't have the time or the resources to do it it might be that you decide you're going to use another kind of a strategy or if you're doing focus groups to pilot tests something often you can do the analysis based on the notes because you've given them three options and they like the middle option but you need to tweak it in these ways and you can often get enough information from your notes to do that kind of analysis and so here we're telling you know a good analysis it's sort of like traveling from here to Chicago there's more than one way of getting there there's some that are faster and some that are longer and there's some that are scenic and some by airplane and some by train when you're doing focus group analysis there's a number of ways of doing it and some involve more risk if you're doing it very quickly and you're not using transcripts there can be a risk that you might overlooked something there are other times though that if you do put in all the time it's going through a transcript it will take you longer if you are gonna do transcripts get a transcription kid off Amazon it is like a little foot pedal and you attach it to your computer and you can play back the recording as quickly or as slowly as you can type and as long as you keep your foot down it keeps going as soon as you lift your foot up it stops and it backs up a little bit so you can keep your hands on the keyboard the whole time and you're not trying to push other buttons it will speed up the transcription an enormous amount and they're just called transcription chits comes with the foot pedal and a little bit of software that and it sounds there's a lot of head nodding so it sounds like you viewers so we recommend those highly so one of the things one of the things that we've we find is that some people you know that when you're analyzing the focus groups what you're doing is you're putting things into categories it's sort of like what we call cleaning the closet or sorting the laundry I think the sorting the laundry is a better one at our household Mary Ann has one way of sorting I have a different way of sorting but we both have to do some sorting of the laundry and it's putting similar things together and and the idea is that when you sort things putting some other things together you you couldn't if you wanted create labels or code I don't do it with mine I'm not sure what my labels would be light and dark no no I think it is like the real scuds II kind of jeans that are full of grease they go in one everything else closer to the other no no yours I found that one that was doing Mary Ann's laundry that no I cannot I've got to look at the fabric now this was a whole new revelation for me fabric I thought all fabric was fabric nope nope there is fabric you have to consider as well as color and so analysis is sort of that is that you're putting things into categories in the literature we call this a constant comparative actually this comes out of the writings of Barney Glasser and Absalom Strauss do you want to explain what that is well kind of comparative is if in order to do this you would have transcripts and so you would start with the very first transcript and you start with the first question you're going to analyze and you take the first quote and you read it and you ask yourself does it answer the question if it doesn't does it answer a different question you move it to that question if it doesn't answer any question you just set it to the side but if it answers the question then you say okay what's that about you put it in a file you take the next quote you read it and you ask yourself the same things does it answer the question yes no does it answer another question and then if it does answer the question you say okay is it is it the same thing as that first quote or is it something else if it's the same you put it in the same pile if it's talking about something else you put it in a new pile you constantly compare the new data to what you already have that's the essence of constant comparative that way there's no cherry-picking you just don't go in and pick out what you like because you're looking at every bit of data and you're saying what's this about and is it do we already have that and then you'll do that you go question by question you do that transcript and then you pull the next transcript in and you do the very same thing the next transcript that group might have been a little different so maybe you've got more categories and then you just keep doing that and then you might get to a point where you say mmm you've got too many categories for this or these are related categories so then you can kind of start playing with the category so you might realize this pile is really big this is really about a couple of different things and so you can tease it apart so it it it is what its name is you're just constantly comparing the new bits of data to what you already have and then you're giving them labels contain basically color saying this is about this and you're giving it a label now what happens when you're doing coding there's a couple things that are important is that this is a process where it's often helpful to have a couple people with you or and it's especially if they were in the focus group itself and when you look at the transcript together with someone else and then you put things into categories and decide what the codes are sometimes you have to you know subdivide the codes sometimes the codes are too broad you don't know when you first start you also kind of keep memos or keep track of what's the logic behind about putting things into different categories so you're consistent as you go through those now then what what happens is you basically there's a couple ways of doing it in in our book and out of the places we talk about a classical method where you're using flip charts and you just cut these things out and you tape them on a flip chart so you visually can see all your comments on this flip chart no so if you have a transcript you can physically cut them apart it's a nice if you've never done qualitative analysis before it's really a nice way to do it because it's a very concrete process and you can visualize what you're doing if you've done it before when I do analysis I'm doing the very same thing only I'm just doing it in word using cut and paste so I'm moving like things together in Word within the transcript so what some people have done is use word and also Excel that if you're familiar with how these programs work Excel has some nice abilities to do some sorting and selection of things that word doesn't have and so what some people have done is they'll start in Word move it to excel that hello and then develop some codes and use that now that is sort of the well for a better word it's sort of the poor person's way of doing it if you're a rich person you can go with the fancy software programs you know such as Apple about I see I believe oh here if you get them at a university they can be not so expensive but if you had to go out and buy those it could be eight hundred nine hundred dollars for software to do that basically what this software does is that you have to still read it you have to give it labels and what the computer does is it puts it in a database and calls it up the computer doesn't tell you what it means you still have to look at that data and say this is what that that coded area what that area means and and you have to describe it the computer won't do that for you and so some people ask the question should I use a computer to help me I you know we what we suggest is that if you're going to do these regularly and often and you're going to use transcripts often then you might want to take a look at some of these computer programs there's a variety of them out there the learning curve is steep it takes you a while to get comfortable with them but we can also tell you that people have done excellent kinds of analysis with focus groups without using these kinds of programs now as you're looking at these programs we should mention just the difference between frequency and extensiveness that sometimes it may be part of our well you should just talk about counting yeah counting you know part of our DNA of being in universities is we tend to think of analysis as a process of counting you know for some reason this is the first thing that comes to mind okay I can just count how many fit this category how many fit that category and then we make interesting assumptions that more means something than less that more comments that's the more important thing you kind of automatically think oh the highest frequency must be the most important thing maybe not maybe no maybe maybe not we you know that's an assumption you have to be careful with in quantitative research that typically the assumption more is better usually there's more insightful than less and oftentimes in qualitative research if you have only one case or a few cases they're considered outliers and we sort of set them aside because they mess up the statistics that when you put them all together it's like in quantitative kind of mess those up and so we set them aside well in qualitative those outliers could be some of the most important bits of information and so we don't set them aside we examine the ones that are on the fringes what do they tell us is it something relevant important that we need to and even if you were doing a study and you had maybe 20 25 people but one person gave a point of view that was different and based on the surrounding things if you felt based on the data you had and the environment and the other factors that there was relevance it may be that you actually discussed that in the report that probably wouldn't be high on your findings but you still may want to discuss it because it brings us some kind of insight to the phenomenon you're looking at I was going to say the same thing that that sometimes it will be that in an entire study something has only said once but but you still kind of hold it up it might be a little Jam and you hold it up and you say you know this was this was sad once but for these reasons we think it's important to pay attention to this like sometimes you're doing a needs assessment and it's only said once but there's only one person who is at the horizon to see that yet coming and so if you're just swayed by the the biggest frequency thing you might be led off track and so we might say you know no one else said this but is this something on the horizon that we better be paying attention to so the whole idea of counting whenever we see focus group reports and there's percentages in them it always makes me a bit uncomfortable like how did they do that because not everybody answers every question so let me give you what are the uses a basis for that let's think about the alternatives other than counting now counting we're not gonna rule that out completely that still has some it might give us insight but let me suggest something else that we call it intensity or passion it's how people say things what what comes along with that communication and that one of the things that we know is that when people feel strongly when there is an intensity of their communication oftentimes it's it actually may be more predictive of behavior than anything else they do that we know that the connection between emotion and behavior is powerful and that emotion tends to sway behavior and influence it in a number of ways and so if you're trying to study and examine why people do things or not do things one of the things to think about is well where was the energy where was the passion were there times in places where people got excited aroused angry now that's that actually may not show up in the transcript I mean it may body language things that are happening in the focus group if if that does happen try to get them to describe it a bit more so that you do have a quote or something what you want is someone to say I am so angry and how this is whatever it is but the quote is something then you can put into the report but also just that what we're often looking for from an analysis standpoint is is the emotion and what you're looking for is there change in how that person is is responding do they get louder to the bang their fists they get softer do they cry does everybody start talking at once so that tells you something another indicator that's going to be help for you is what we call specificity that means how much detail are they able to give you on what they're talking about and that when people have had a very direct experience and a vivid experience oftentimes they remember the details and so as you're asking someone they're talking about something's they tell me more about it how do they come how do they talk about it when you ask that do they give you chapter and verse do they give you exact kinds of details or do they talk in a very general way now it's not that you discount the general but when you hear specific your ears perk up and you listen very carefully because this had a vivid impression on them and it's something very likely very connected to their being in their behavior so when we hear the kind of specific concrete from an analysis standpoint we give that more weight so these are things you know that you need to weigh a number of things not just how often or how many different people intensity specificity and these are some of the things that you can that you have to sort of be there in the group that's that is one of the keys to analysis let me tell you that one anecdote about we were doing something a couple years ago and one of the suburbs here of st. Paul the issue was on solid waste disposal and should you have one garbage hauler or many garbage haulers in the communities st. Paul now is facing that very same issue at a number of communities what we discovered we were asked by Washington and Ramsey County to go into some of the communities and do focus groups what surprised us where some of the residents came in who were very emotional about this topic no in our neighborhood we're not too emotional about garbage haulers I mean we just like it to be gone and that's a good idea and there are many garbage trucks going up and down our alley but what we found in these groups are people that were pounding their hands on the table talking about how important it was for them to hire their own personal garbage hauler that was an American freedom that they wanted to preserve and they didn't want it taken away by some level of government we were a bit surprised at that the advice we gave to the communities were back away from this one because the emotion is too intense people are going to fight this issue some many most of the people could care less but there was a strong faction the same thing is happening right now in st. Paul and other communities that are studying it and partly it's because garbage haulers are sort of agitating and they're they're sending out messages to their customers saying you want to be sure to hire your own personal garbage hauler because this is an American freedom that people have fought and died for to that go they actually brought in letters and like the borders for American flags and yeah affect your freedoms and you know it's like that was a very important discussion but what's what's important about this is if you are in those focus groups you got a kind of visceral feeling about the amount of energy and passion people add around topic if you were trying to do analysis just from the transcripts all that's gone right you you have no idea like that they were pounding their fists and you know sure you the transcriptionist could put that in there it's not the same as having been in the group so what we recommend is whoever's doing the analysis should be in as many of those groups as possible because things happen we've got a friend who does a lot of focus groups around public health things she actually says that she feels as though they haven't gotten to the heart of the matter until somebody cries now we're not trying to get people to cry in ours but it is something to pay attention to know that you probably are getting it and you know that's one of the uniquenesses and focus groups compared to other forms of research I mean I've never seen anyone get angry or cry over a survey or an individual interview I mean maybe they got angry but but in a focus group in it I mean I don't mean to say that it happens all the time but it is interesting that every once in a while there is an emotional overtone and that is an important bit of information that's very useful if you're trying to design programs put things together it could be things that people are fearful of people that are concerned about any number of things so that's kind of the nuts and bolts of analysis let's stop now and see if there's some questions on analysis over here you got one oh she had earlier brought up this idea that when you do focus groups you often raise expectations that you're actually gonna do something this is important because if you're working in communities and you keep going out and asking them for information and they don't see that anything's different or don't get a follow up they don't know what you did they begin to not want to play your game anymore right so one of the things that we recommend is that when you are doing focus groups keep track of who who shows up and then what we do is send out a thank you letter or we call it a thank you report and so we'll send out and often again we want to have this signed by somebody who they know and respect within the organization but we say thank you Roberta's throat for participating we heard you here are the top three things that we've learned based on that here's what we're gonna do we also heard this and right now we can't do anything about that kind of explain that especially if there was some big thing that came up and you can't do anything about it and then we'll say we've included an executive summary of the report if you want more information here's the name and phone number of somebody to contact and again thanks for participating know that's helpful to me though I've gotten burned on this one and there are times where organizations I mean I've been asked by you know Hennepin County Health the state of Minnesota foundations the University of Minnesota to all go go into indigenous communities and ask questions and there's been no follow-up and my new philosophy on this is that before I say yes to any of them I'm going to extract a promise that they're going to monitor it and stay in touch in other words it isn't just the letter afterwards but that on a regular basis somebody stays in touch with that group and tells them what progress because I know that like when the universe that he did something with the American Indian community that we did this big less thing nothing ever happened now the University was studying heck we like to study things for years before anything happens and that was what was happening and they interpreted that as being indifference and it wasn't the immediate report it was two years three years later somebody should have been in touch with them and said we're continuing to look at this and we'd like to have more conversations that and so some of these if you're working for an organization get people high enough up to say we're gonna make a commitment if we're going it's not just a one-shot band-aid okay we've done our dues now - whatever it is it's a commitment if you're gonna start it and you carry it through not just today but next year and the year after words and I think that's how you build trust within a community instead of fly-by-night which so often we do and then they get ticked off when nothing happens I don't blame them yeah okay and then let's just question back here yeah well I would encourage you to keep thinking about this because this is a good topic one of the things you might use and we've thought about it from time to time is that scale that your physician uses on how much pain do you feel on a ten-point scale where's your pain today and there are times where if I would see someone showing some kind of an emotion I would say where is that let's say that I'm not sure I can't think fast enough to do it but I would have probably used that on these people with solid waste is like okay if you're talking about your frustration with having one hauler where would you place that with 10 being really frustrated than one I don't you know I don't care where would you be sort of emotionally on a ten-point scale I don't know I've not done it enough to know if that would work but something like that where I can go back and say okay this ranks a 7 on a 10-point scale I think that kind of insight to a reader would say oh I better pay attention to that all I can tell you is up in Maplewood there's a bunch of mostly men older men who are really enraged on this topic I don't know how many I don't know you know what percentage it would be but I do know the logic they use and I don't think they're going to change very easily they seem to be really dug in and I would bet the same as happening in Saint Paul that the people are locked in a position and it's going to be a question of how do we get people to move on this or how do the city leaders move forward when you have such strength but I keep thinking about how you might do it because I think that if you can come up with a strategy that would be very insightful any other yes yes yes one of the things we will have someone watch for is like head nods but usually the moderator will try to will see the head nods and we'll try to get someone to verbalize it like I I see some head nods with someone like the comment on that and because what you would like to do is to get a verbal comment more than get it out of just banging into verbal things so that you can use it in a report just saying someone is waving their hands gesturing wildly may not be enough that is your it's back to your point it's your interpretation of what that means and you could be totally wrong so you as much as you can as the moderator what you're trying to do is have them talk about it you

Channel: UMN_MCH

Views: 5,993

Keywords:

Id: 5sMe_iQfd0s

Length: 41min 4sec (2464 seconds)

Published: Tue Jul 16 2019