Qualitative Measures

Qualitative research is a vast and complex area of methodology that can easily take up whole textbooks on its own. The purpose of this section is to introduce you to the idea of qualitative research (and how it is related to quantitative research) and give you some orientation to the major types of qualitative research data, approaches and methods.

There are a number of important questions you should consider before undertaking qualitative research:

Do you want to generate new theories or hypotheses?

One of the major reasons for doing qualitative research is to become more experienced with the phenomenon you’re interested in. Too often in applied social research (especially in economics and psychology) we have our graduate students jump from doing a literature review on a topic of interest to writing a research proposal complete with theories and hypotheses based on current thinking. What gets missed is the direct experience of the phenomenon. We should probably require of all students that before they mount a study they spend some time living with the phenomenon. Before doing that multivariate analysis of gender-based differences in wages, go observe several work contexts and see how gender tends to be perceived and seems to affect wage allocations. Before looking at the effects of a new psychotropic drug for the mentally ill, go spend some time visiting several mental health treatment contexts to observe what goes on. If you do, you are likely to approach the existing literature on the topic with a fresh perspective born of your direct experience. You’re likely to begin to formulate your own ideas about what causes what else to happen. This is where most of the more interesting and valuable new theories and hypotheses will originate. Of course, there’s a need for balance here as in anything else. If this advice was followed literally, graduate school would be prolonged even more than is currently the case. We need to use qualitative research as the basis for direct experience, but we also need to know when and how to move on to formulate some tentative theories and hypotheses that can be explicitly tested.

Do you need to achieve a deep understanding of the issues?

I believe that qualitative research has special value for investigating complex and sensitive issues. For example, if you are interested in how people view topics like God and religion, human sexuality, the death penalty, gun control, and so on, my guess is that you would be hard-pressed to develop a quantitative methodology that would do anything more than summarize a few key positions on these issues. While this does have its place (and its done all the time), if you really want to try to achieve a deep understanding of how people think about these topics, some type of in-depth interviewing is probably called for.

Are you willing to trade detail for generalizability?

Qualitative research certainly excels at generating information that is very detailed. Of course, there are quantitative studies that are detailed also in that they involve collecting lots of numeric data. But in detailed quantitative research, the data themselves tend to both shape and limit the analysis. For example, if you collect a simple interval-level quantitative measure, the analyses you are likely to do with it are fairly delimited (e.g. descriptive statistics, use in correlation, regression or multivariate models, etc.). And, generalizing tends to be a fairly straightforward endeavor in most quantitative research. After all, when you collect the same variable from everyone in your sample, all you need to do to generalize to the sample as a whole is to compute some aggregate statistic like a mean or median.

Things are not so simple in most qualitative research. The data are more “raw” and are seldom pre-categorized. Consequently, you need to be prepared to organize all of that raw detail. And there are almost an infinite number of ways this could be accomplished. Even generalizing across a sample of interviews or written documents becomes a complex endeavor.

The detail in most qualitative research is both a blessing and a curse. On the positive side, it enables you to describe the phenomena of interest in great detail, in the original language of the research participants. In fact, some of the best “qualitative” research is often published in book form, often in a style that almost approaches a narrative story. One of my favorite writers (and, I daresay, one of the finest qualitative researchers) is Studs Terkel. He has written intriguing accounts of the Great Depression (Hard Times), World War II (The Good War) and socioeconomic divisions in America (The Great Divide), among others. In each book he follows a similar qualitative methodology, identifying informants who directly experienced the phenomenon in question, interviewing them at length, and then editing the interviews heavily so that they “tell a story” that is different from what any individual interviewee might tell but addresses the question of interest. If you haven’t read one of Studs’ works yet, I highly recommend them.

On the negative side, when you have that kind of detail, it’s hard to determine what the generalizable themes may be. In fact, many qualitative researchers don’t even care about generalizing – they’re content to generate rich descriptions of their phenomena.

That’s why there is so much value in mixing qualitative research with quantitative. Quantitative research excels at summarizing large amounts of data and reaching generalizations based on statistical projections. Qualitative research excels at “telling the story” from the participant’s viewpoint, providing the rich descriptive detail that sets quantitative results into their human context.

Is funding available for this research?

I hate to be crass, but in most social research we do have to worry about how it will get paid for. There is little point in proposing any research that would be unable to be carried out for lack of funds. For qualitative research this is an often especially challenging issue. Because much qualitative research takes an enormous amount of time, is very labor-intensive, and yields results that may not be as generalizable for policy-making or decision-making, many funding sources view it as a “frill” or as simply too expensive.

There’s a lot that you can (and shouldn’t) do in proposing qualitative research that will often enhance its fundability. My pet peeve with qualitative research proposals is when the author says something along these lines (Of course, I’m paraphrasing here. No good qualitative researcher would come out and say something like this directly.):

This study uses an emergent, exploratory, inductive qualitative approach. Because the basis of such an approach is that one does not predetermine or delimit the directions the investigation might take, there is no way to propose specific budgetary or time estimates.

Of course, this is just silly! There is always a way to estimate (especially when we view an estimate as simply an educated guess!). I’ve reviewed proposals that say almost this kind of thing and let me assure you that I and other reviewers don’t judge the researcher’s credibility as very high under these circumstances. As an alternative that doesn’t hem you in or constrain the methodology, you might reword the same passage something like:

This study uses an emergent, exploratory, inductive qualitative approach. Because the basis of such an approach is that one does not predetermine or delimit the directions the investigation might take, it is especially important to detail the specific stages that this research will follow in addressing the research questions. [Inset detailed description of data collection, coding, analysis, etc. Especially note where there may be iterations of the phases.]. Because of the complexities involved in this type of research, the proposal is divided into several broad stages with funding and time estimates provided for each. [Provide detail].

Notice that the first approach is almost an insult to the reviewer. In the second, the author acknowledges the unpredictability of qualitative research but does as reasonable a job as possible to anticipate the course of the study, its costs, and milestones. Certainly more fundable.