The Qualitative Research Method

My final blog looks back to the one of the earliest discussions on the different approaches to qualitative research methods and reflects on what the approaches mean to the broader methodology of qualitative research. Glesne reviewed several different approaches in qualitative methods, including among other things, case studies, ethnography, and grounded theory. Although the different approaches are ostensibly tied together by a higher-order paradigm, my proposition after the past months is that there is a noticeable fissure among qualitative researchers far more pronounced than in the quantitative methodology. Because quantitative researchers are united under the clear doctrine of the scientific method, there is no confusion about how to approach research in the assumptions or interpretations made. Although postpositivist acknowledge that a complete separation between researcher and research is only a regulatory ideal and cannot feasibly be accomplished, this ideal is consistent among researchers. However, I perceive qualitative research methods to be a fragmented as the topics it studies. I see no continuity in the values or beliefs of the method apart from some distally accepted notions of subjectivity and constructivism that do not permeate research. Indeed, I found myself identifying with the perspective of grounded theory researchers and see a lot of conceptual overlap between our ideals. However, I see fundamental disagreement between my values with that of an autoethnographer on account of the subjectivity. So, should we try to package qualitative research as a “thing” in itself or is it a vapor methodology that exists on the cover of textbooks and in the minds of interpretivists.

I admit that my knowledge of qualitative research is cursory at best, but I fail to find the single thread that holds qualitative research methods together. Even in this class, I found the pile sort, free list, and rank order activity to be entirely fruitful because it closely aligns with the theory construction paradigm to which I have grown accustomed. I even understand the necessity of interviews and focus groups, but I fail to see how all of these approaches are housed under a broader label of qualitative methods. I see distinct departures in qualitative methods on important values, such as generalizability, thematic construction, and theoretical abstractions between traditional ethnographers and grounded theorists. I fear the qualitative method loses some traction in society because of what has been termed the jingle-jangle fallacy. That is, different concepts have been described using the same word, and the same concepts have been conceptualized using the different words. In the case of research methods, qualitative design is used to capture all non-quantitative research no matter how different its approach. I think there warrants further discussion on whether the term qualitative has any intrinsic meaning or if it is a moniker used to represent everything and, at the same time, nothing. I have become growingly fond of qualitative designs over the last months, trying to imagine myself engaging in research using one of the submethods of qualitative research, but I would like greater precision in the use of qualitative methods if I am to talk to a quantitative colleague about a project on which I am presently working.

Making Explicit the Implicit

Understanding of qualitative research methods over the course of this class has not for me been linear. Indeed, I have traveled through a cyclical pattern wherein I feel like I have a strong understanding of the materials one day, and the next day my perceived mastery over the content has all but disappeared. I suspect that the nonlinear trend is due, in part, to reading the texts. I often become caught in the ennui of inconsistency and get lost in micro-level descriptions, ignoring the forest for the trees.

In looking at the broader picture of qualitative research, however, I am often confused about the principal goals of the interpretation. Is explanation the prevailing objective or is it prediction? I, for long, thought that it was the former rather than the latter; I did not suspect, given the ontology and epistemology of qualitative research methods, that prediction or generalization was at all the goal of the research. Maxwell’s (2013) treatment of generalization in qualitative research therefore surprised me. Drawing his ideas largely from the work of Becker (1991), Maxwell contends that “this does not mean qualitative studies are never generalizable beyond the setting or informants studied. The most important reason for this is that the generalizability of qualitative studies is usually based not on explicit sampling of some defined population to which the results can be extended, but on the development of a theory of the processes operating in the case studied, one that may well operate in other cases, but that may produce different outcomes in different circumstances” (p. 138). What? How can this at all be true? Although I realize the far departure of qualitative methods from Karl Popper’s notion of falsifiability, can Maxwell’s proposition about operating parallel processes but producing different outcomes be unsupported? The answer seems to be emphatically no.

I can somehow construe abstractly that Action A and Action B have some conceptual processual overlap and with different outcomes, but this should not mean that the information I gather in research on Action A can be at all generalized to Action B. I think Maxwell and other methodologist like Becker need to acknowledge that nothing is ever generalizable in qualitative research because the results are bounded by time and context. I don't see this as being a limitation in any regard like Maxwell seems to believe as evidenced by his vehement "defense" of qualitative research (see p. 136-138). Instead, I see the lack of generalizability as consistent with the metatheoretical traditions underlying the qualitative research method. If indeed generalizability and attendant prediction were so important in qualitative research methods, then why are we not seeing an larger emphasis placed on replication of study findings that we see in postpositivist tradition?

What Does Validity Have To Do With It?

The topic of validity causes great internal consternation among qualitative researchers. Maxwell (2013) articulates the evolution of historical arguments for or against recognizing validity in the social constructionist tradition of conducting research. Some abandoned validity as a necessary component of investigation, arguing that it was too closely tethered to postpositivist approaches to research. Although Maxwell and I disagree on how the term “validity” in research should be construed, I agree there is an inherent need to rethink the role of validity in qualitative research.

Maxwell (2013) broadly characterized validity as the “the correctness or credibility of a description, conclusion, explanation, interpretation, or other sort of account” (p. 122). Because a subjective or objective truth never emerges in research, the “correctness” criterion is only highly theoretical in nature and can never be accurately estimated with any level of precision. The“credibility” criterion (i.e., truthfulness and functional expertise of the research) too liberally confounds validity with methodology. Instead, I would advocate for a view of validity in research as the extent to which our conclusions or explanations are what we say they are. Although this may appear to be a nuanced difference between Maxwell’s definition and the one presented here, I would contend that the differences are critical.

Validity is an inherently polarizing artifact in qualitative research because the inconsistent applications of different methodologies implicate validity indiscriminately. In strict constructionist ontology and subjectivist epistemology, a recognition of the constant flux of phenomena in a given participant is exercised. The dynamic variance of interpretations, explanations, and descriptions on any topic is captured at the exact moment of the interview between a specific researcher and the subject under investigation. The results naturally cannot extend to other situations or scenarios given that they are subjective interpretations contained or delimited by time. So, validity is only an existential issue because, so long as all biases and anomalies of the research are made apparent, the research is considered“valid” at that time point between the two interactants. The issue at hand is whether validity means anything under these conditions.

One could argue that because qualitative researchers make interpretation based on their own subjective experiences with the respondent at a given time, their subjective interpretation, in every instance, meets the established criteria of validity. Indeed the disseminated results are reporting what they are purportedly reporting for the given researcher at the given time. No one then but the researcher can say that the results or the methodology are invalid because even if something were inconsistent with previous literature (i.e., externally inconsistent), the anomaly may have happened at the time unique to the relationship fostered between the subject and researcher. So, we can expect that case studies and other small sample qualitative methods make a lot of what quantitative researchers would call“Type 1 error” because no systematic consistencies can be ascertained. Type 1 error asks the question of whether the results in the sample are consistent with reality. Again, this is likely not a concern for strict constructionist researchers because the sample need not mirror an objective reality given that no objective reality exists. So, what does validity have anything to do with research critique in qualitative methods apart from something researchers should abstractly consider at the outset of a study?

Identifying and Dealing with Validity Threats (Exercise 6.1)

Validity is something that all researchers must consider when conducting research. In a sense, validity poses the question, “Are we sure the results that we conclude from our observation in fact mirror reality?” Many issues in research can delegitimize the internal and external validity of research. A careful consideration of certain aspects of data collection before observation can mitigate many of the concerns relevant to internal and external invalidity in research.

Many concerns related to validity arise and must be considered at the fore of research projects that examine the topic of cyberbullying. Because bullying is considered an antisocial behavior for both bully and victim, social desirability is something that must be appreciated prior to research. Bullies may deny engaging in behaviors, rebuff the intent of their actions, or artificially trivialize the effect the bullying had on the victim. They may also refute the reflexivity of bullying event. Victims might also minimize the effects of the bullying as a display of strength and to overcome intrapersonal feelings of victimization. To handle threats of social desirability, I could frame the goals of the research project as how they think the “generalized person” who is cyberbullied would be affected by the experience. Depersonalizing the questions allow for more honest answers given that the responses do not implicate the self in relation to the event.

Reactivity might be a validity threat given that the amalgam of research demonstrates children and adolescents’ unwillingness to consult adults for advice or help on cyberbullying victimization. It may be that people will not speak openly about the experience because they fear reprisal from the bully or limitations to their freedoms (e.g., limited cell phone use) by their parents. The goal early in a semi-structured interview would be to engage in trust-building exercises that talk about less “dangerous information” and move toward more sensitive topics. In this way, children and adolescents will be allowed to warm up to the occasion and the researcher.

Lastly, in following a cohort of victims over time to investigate the longitudinal effects of the cyberbullying experience, researchers must consider maturation of the participants in interpreting the effects over time. As children and adolescents grow older, their long-term ability to cope with the victimization is confounded by the natural processes they are going through as they become older. In interpreting data collected from children and adolescents on psychological and social behaviors, it is necessary to recognize what has become known from the developmental psychology literature on maturation and interpret the study’s results aside these findings.  

Reflecting on Your Research Relationship (Exercise 5.1)

I am of the mindset that the relationships a researcher fosters with his or her targets of analysis are the single most important attribute of qualitative research methods. Whereas personality and social skills, things cultivated through life, do not much matter in quantitative research, the ability to obtain data as a bad interviewer is indeed a monolithic obstacle almost impossible to overcome in qualitative research. With practice and through rote memorization, individuals can eventually learn how to compute an ANOVA, simple linear regression, or t test, but it takes an almost innate sensibility to become a good qualitative researcher, something that cannot really be learned in a class or practiced. Courses like this teach skills and technique but do not teach someone how to create rapport with subjects given the indefinite nature of an interview. Although the general sentiment among multiple methodologists I have spoken to seem to be that qualitative research is not more difficult than quantitative research but certainly more time consuming, I would now argue that qualitative research is both more difficult and time consuming in light of the myriad intangibles (e.g., social skills, immediacy) necessary to be successful as a qualitative research methodologist.

The way I present myself to the respondents is an important first step in my venture to build rapport. To put participants at ease, I would select clothes that appear professional but not so much so that it creates a noticeable imbalance between what I am wearing in comparison to them. I would select clothes appropriate to the setting. I would then consider the setting in the room. I would likely organize multiple chairs in sequence, from near to far, allowing the participant to choose their level of physical distance when answering the interview questions.

 Status and power are two important things to consider given that they are practically unavoidable in my research. Because my only pathway to getting data from children and adolescents is through the Department of Education, I am obligated to present my affiliation to students, who then form associations with my position at the University that invariably bias their responses. I have, in the past, tried to mitigate the “outsider effect” and the power difference between participants and me by substitute teaching a few times before the data collection. The repeated contact in a classroom setting appears to disassociate the source credibility from the source for children and adolescents. The way in which I come off as a nice substitute teacher or the mean substitute teacher will also influence how much people will confide in me. I found marked differences in willingness to response to questions when I came to the school as a sterile outside researcher in comparison to when I would first substitute teach at the school and then present my other research intentions.

 The principal ethical consideration I must make is whether the position of a substitute teacher makes inflexible children and adolescent’s idea of voluntary participation. Some students may feel coerced or compelled to complete the study because they come to the conclusion that participation is a component of their grade. To hedge against this potential ethical concern, I would ensure that participants realize they can choose whether they would like to volunteer their time for the study, stressing the notion of volunteerism, and telling participants about their research rights. All of the steps taken in creating a relationship with your target respondents are crucial in the quality and quantity of data researchers get from their data collection methods.

Developing Your Research Question (Exercise 4.1)

I came to the conclusion that the focal interest of my study, after reviewing my concept map, is that of the unknown as it relates to cyberbullying. Although I rarely build the rationale of my research on arguments about “how we know little about a phenomenon,” I seem to gravitate toward studying novel concepts rather than replication in this research domain. Perhaps this is an artifact of my latent ill regard for research and theory on cyberbullying, which has been fraught with conceptual ambiguities and inconsistencies. My guiding research question therefore is formative in a sense. 

The conceptualization of cyberbullying in the literature is clear, and a fair amount of agreement exists among scholars, practitioners, and even those enmeshed in cyberbullying episodes about the characteristics of cyberbullying. Indeed, cyberbullying can be considered an intentional hurtful or embarrassing message repeatedly transmitted over a digital medium between people of different power. The definition is noticeably multifaceted in that it comprises complex ideas such as intentionality, repetition, and power imbalance. But little attention has been paid to explicating all these components in research.

 The central focus of my study appears to be the power imbalance experienced between the bully and the victim. The concept map provides some clarity in this domain. All ideas seem to want to trace back to the power imbalance generated in cyberbullying because it is a concept so imprecise and veiled in uncertainty. My research question is undoubtedly related to the power imbalance. The things most interesting to me center around two different questions:

1.       What do child and adolescent victims of cyberbullying say affected their perceptions of power imbalance between the bully and them?

2.       Did the perceptions of power imbalance influence the type of negative outcomes experienced on account of the cyberbullying?

I was sure to include some indication about the limitations of the purposive or snowball sampling I am likely to employ with a project of this nature. Randomly sampling would not get the necessary population of individuals I need for this sample. Sampling bring into light further questions about the methods to which I can relate these questions. To answer Research Question 1, I can use a number of different methods but the one I feel might best suit my needs is the rank order, free list, and pile sort method. In scrutinizing my questions, I feel that this method is germane to locating the cultural domains that make up the power imbalance. I can ask victims: “What kinds of the things cyberbullying victims think affects the differences of power between bullies and victims communicated through digital devices?” The question will elicit cultural domains of power imbalance in cyberbullying experiences.

The methods used to test Research Question two would be more complicated given the goal of finding out information about the nature of the relationship between power imbalance and negative outcomes. First, it should be made clear what negative outcomes are being implicated at the outset of the project. Some unanswered questions right now seem to be whether I am looking at cognitive, emotional, or behavioral outcomes and are they inherently negative or can they be positive as well? I would use in-depth interviewing to answer these questions and probe the nature of the relationship between power imbalance and these outcomes.

The Wheel of Science

I understand and appreciate the lens through which a researcher with a subjectivist epistemology approaches research. The subjectivist approach to research entails interacting with targets of interest in their social contexts to understand social phenomena. Objectivism, on the other hand, seeks to make generalizations about social phenomena for explaining, predicting, and later controlling the phenomena. Common to these epistemological poles is the notion that researchers from either approach seek abstractions among people’s nuanced behaviors.

Abstractions are generalizations that form the building blocks for theory. Themes develop through objective and subjective observation of behaviors, and researchers begin to make preliminary guesses about how these abstractions relate through laws of interaction and formal propositions. The development of these abstractions, however, differs by the epistemological standpoint from which research is conducted. In the subjectivist tradition, the researcher becomes the tool through which data are gathered. A separation between the researcher and the research is neither a focal concern nor a regulatory ideal. Although the biases of the research should be made transparent, these biases do not hurt the systematic process of science. The objectivist tradition relies on the scientific method to maintain objectivity in finding truth.

In several research method texts, a “wheel of science” is publicized wherein research created from the subjectivist and objectivist work hand in hand to develop and extend theory. Although I appreciate the idea of the wheel of science, the notion of a complementary system of empiricism in practice does not exist beyond multiple methodologists. How researchers practice research is largely guided by the departments in which they learned research methods. When they become indoctrinated into a specific view of science, researchers enter into silos, with neither side reading much of the output of the other. I, only on rare occasions, read qualitative research because I have been taught to believe that results from qualitative research are tenuous at best. My research previous to this class was thus influenced only modestly by data-first inquiry unlike what this wheel of science presents as almost commonplace. And although my initial reaction is that it might be a product of my educational upbringing, I recently searched through the flagship journal of our discipline only to find no qualitative studies and only a handful of citations of qualitative studies in the references of these quantitative investigations.