“Evidence-based medicine builds upon, rather than disparages or neglects, the evidence gained from good clinical skills and sound clinical experience.” (Sackett, 1995: p. 840)
Qualitative research methods are used to collect and analyse data that cannot be represented by numbers. This paper aims to explain the different practices of data collection and analysis found in qualitative research and to outline when it is appropriate to use these methods. We hope that the reader will gain confidence in the critical appraisal of published research that uses qualitative methods.
Qualitative v. quantitative research
The methods used in qualitative research are not new and have a long tradition in a variety of academic disciplines (Reference Murphy, Dingwall and GreatbatchMurphy et al1998). What has emerged over recent years is the bundling together of these miscellaneous techniques under a single heading, often for the purposes of contrasting them with quantitative methods. The qualitative/quantitative split has become pronounced in health sciences, encouraged by the hierarchy of research methods set out by the evidenced-based medicine paradigm (Reference Sackett, Richardson and RosenbergSackett et al1997). As a result, qualitative research has become negatively defined as “what quantitative research is not”. Thus, many have a ready-made opinion about qualitative research based on their perception of quantitative methods: usually either deep mistrust or blind enthusiasm. Neither of these positions is particularly valuable if founded on misunderstanding.
The absurdity of defining qualitative research through its opposition with quantitative methods quickly becomes apparent. Qualitative research is and must be based on empirical evidence or it cannot claim to be research. The difference is that the evidence (the data collected) is not in numerical form and requires interpretative rather than statistical analysis. It is equally absurd to argue that data obtained in numerical form are not subject to an interpretative process while being collected. For instance, one measures depression on a recognised, reliable scale that translates the subject's feelings into a numerical format.
The hierarchy of research methodologies used in evidence-based medicine is ranked in the order of suitability for evaluating outcomes. There can be little argument that the best way to answer the questions “What works?” or “Is this drug better than that drug?” is by doing a randomised controlled trial. However, the proposition “good research is done by doing a randomised controlled trial” does not necessarily follow (Reference NaylorNaylor, 1995). The purpose of a research method is to answer a research question. By narrowing the repertoire of methodology, the number and type of questions that can be answered are similarly narrowed (Reference BlackBlack, 1996). Rather than changing the research question to fit the method it is better to select the appropriate methods to answer the question (Reference Green and BrittenGreen & Britten, 1998). Mental health researchers appreciate the value of methodological eclecticism (Reference Marshall, Powell and LloydMarshall et al1996).
Differences between qualitative and quantitative methods
One of the drawbacks with data that are not easily reducible to numbers is the cumbersome nature of the material collected. One hour of recorded conversation may take 3–5 hours to transcribe and result in many pages of text for analysis. Although in principle there is no limit on sample size in qualitative research, time and resource constraints make large samples impractical. This leads to a major criticism of qualitative studies: that the data lack generalisability and are unreproducible, as they are context specific. However, this weakness becomes the strength of such methods when used appropriately. The point is always “What is the question you are seeking to answer?”
Quantitative researchers seek a large random sample that is representative of the general population. The purpose is to eliminate individual variations and focus on generalisations. This allows the statistical inference of results and conclusions that are applicable across the entire population. Thus, a randomised controlled trial of treatment A can reliably conclude that it is of more benefit to a greater number of people than treatment B.
Qualitative researchers seek a small detailed sample to produce a plausible and coherent explanation of the phenomenon under study. The purpose is to examine a phenomenon or interaction and to understand it. The results are not usually statistically generalisable, although the theory generated may be. Hence the term ‘theoretical generalisation’.
Theoretical generalisation
An example of qualitative research methods resulting in useful theoretical generalisations can be found in the rich observations of behaviour and detailed reporting of patient's accounts by Jaspers. Phenomenology, which Jaspers defines as “the systematic study of subjective experience” by “representing, defining and classifying psychic phenomena” (Reference JaspersJaspers, 1968), uses methods that would today be included under qualitative research. Although Jaspers's theoretical generalisations have been developed and refined over time, they have undoubtedly been of value. Another classic example of theoretical generalisation arising out of the detailed observation and in-depth interviewing of a small sample are Freud's individual case studies (Reference Freud, Strachey and RichardsFreud, 1977). Freud's work may not be as rigorous as qualitative researchers today would expect (see Reflexivity), but he nevertheless developed the ‘science of psychodynamics’, using an iterative process with constant feedback between theory and observation, meticulously recorded (Reference KvaleKvale, 1999).
Sampling
Sampling in qualitative research is described as non-probabilistic or purposive, as subjects can be chosen deliberately in order to test a particular theoretical premise. The purpose of sampling here is not to identify a random subgroup of the general population from which statistically significant results can be extrapolated, but to identify cases that possess relevant characteristics for the question being considered. This process should nevertheless be systematic and not based on convenience.
Theoretical sampling is a type of non-probabilistic sampling where the objective of developing a theory guides the process of sampling and data collection. A case is selected because it is expected to exemplify or test some identified theoretical issue. The resulting data lead to refinement of that theory and guide further data collection; the relation between sampling and explanation is thus iterative and theory led.
When to use qualitative methods
Qualitative research methods are particularly suited to answering the question “How does this come to happen?” (Box 1). In the exploratory stages of a research project, qualitative methodology is used to clarify or set the research question, to aid conceptualisation and to generate a hypothesis. This method has been much used in psychiatry, in the reporting of clinical observations from practice, often by means of a case study or series. Examples are Edwards & Gross's (1976) proposal of the features of alcohol dependence syndrome or Russell's (1979) classification of eating disorder syndromes based on a series of 30 cases. Both studies demonstrate some of the principles of good qualitative research: systematic observations reported with detailed description, from which generalisable theories can be obtained and further hypotheses generated.
When to use qualitative methods
During the exploratory stages
Investigating anomalies
Examining policy implementation
Collating user views
Qualitative studies can also be useful following quantitative work to interpret, qualify or illuminate findings. This is particularly helpful when focusing on anomalous findings, as they test the main hypothesis formulated. For example, Morgan (1979) identified a small series of psychiatric in-patients who, despite appropriate intervention, went on to complete suicide. Through an analysis of observation of practice on the wards and informal interviews with staff he developed the proposition of ‘malignant alienation’ to describe features in the relationship between staff and patients that were found in this anomalous group.
Clinical policies, although based on the best available evidence, often run into problems in practice. Qualitative methods can be useful in examining exactly where these problems arise and what might be done to avoid them. Unfortunately, this exercise is often undertaken only after an adverse event has occurred. Inquires after homicides essentially use qualitative methods such as documentary analysis and interviews to investigate individual cases where policies have failed. Another important input into clinical policy development are the views and experience of user groups. Qualitative methods of interviewing, such as focus groups, are useful for in-depth examination of issues with user representatives as an adjunct to comprehensive surveys.
Qualitative methods
Methodologies associated with qualitative research can be categorised under three main headings: observations, interviews and the study of written records (Box 2). Although presented separately here for clarity, qualitative methods are often used in combination, such as interviews and observation, or interviews and examination of written records. This is called ‘triangulation of sources’ and it enhances the completeness of the data (after all, what people say they do in an interview setting and what they actually do as revealed by observation can be quite different).
Methods of qualitative research
Observation
Unstructured or structured
Observer's role: participant or complete
Interviews
In-depth, semi-structured or structured
Interviewees: individual or group
Study of written records
Informal documents (e.g. diaries, letters) or formal documents (e.g. case notes)
Observation
Observation can be either unstructured, where the relevance of actions and particular events emerges gradually over time; or structured, using pre-established observational schedules that determine when and what is observed and recorded. Unstructured observation attempts to record behaviour with as few preconceived ideas as to what is happening as possible, gradually making sense of what is going on from the experience of being in the setting. Structured observations use existing theories as a framework to guide observational recording; this saves time, but runs the risk that assumptions are made.
The role of the researcher while conducting an observation is important to the type of data collected (Reference Mays and PopeMays & Pope, 1995). Observers can participate to a greater or lesser extent in the activities they are observing and record observations (field notes) at varying intervals. This form of observation means that interactions are part of the process, which blurs the distinction here between observations and interviews. Participant observation has a long tradition in social anthropology (Reference MalinowskiMalinowski, 1922). It can be covert, where the researcher hides his or her true purpose and identity from the subjects, or overt. There are ethical problems with covert observation, particularly in health service settings. Complete observation means that the researcher makes no attempt to interact with events other than to record them.
Interviews
As with observations, interviews can be classified according to their degree of predetermined structure (Reference BrittenBritten, 1995). Structured interviewing consists of administering questionnaires; the researcher may have been trained to ask the questions in a standardised manner and responses to questions may have to be from a fixed selection. Semi-structured interviews have various fixed schedules of questions, but also allow the interviewer some choice in selecting the lines of questioning to take. Responses are open-ended and can be explored in detail; they can be recorded during the interview either by written note-taking or on audio- or videotape.
In-depth or ‘long’ interviews may have a pre-set theme, but the interviewer and subject are free to respond and explore whatever issues they identify as relevant. Interviews are recorded on video- or audiotape and later transcribed. An example of the insights gained through in-depth interviewing can be found in the works of Tony Parker. He conducted many hours of interviews with a single subject, then edited the transcripts to produce narratives about individuals lives (Reference Soothill and ParkerSoothill & Parker, 1999).
Focus groups
Interviews can also be conducted in group format. It is useful to have some predetermined structure for such groups in order to provide a focus for discussion (hence the term ‘focus groups’;Reference KitzingerKitzinger, 1995). Focus groups contain a small number of members selected to give a range of views on the chosen topic; they conduct a semi-structured discussion guided by key points or questions and facilitated by two or more recorders. Sessions can be audio- or videotaped. It is important when using audiotape to make careful records of who is speaking at all times. Other aspects of the group's behaviour can also be observed and form part of the data. Focus groups are particularly valuable in gathering user views on service provision (Reference Powel, Single and LloydPowel et al1996).
Written records
Documentary analysis is not solely the realm of the medical historian. It can be undertaken on either formal records such as case notes and death certificates, or informal records such as diaries and letters. In a qualitative study, a combination of documentary analysis, interviews and observation is often used. Barret's (1996) ethnography of a psychiatric unit is a good example of such a study.
Analysing qualitative data
The analysis of qualitative data appears to be very different from quantitative analysis. Instead of testing hypotheses in a sequential manner using a series of statistics, analysis of qualitative data runs concurrently with data collection. The purpose of data collection is accurate representation of the phenomena under study using detailed or ‘thick’ description (Reference GeertzGeertz, 1993). As an account emerges, categories and themes become apparent, and it is this generation of theory from the data, rather than the testing of a prior hypothesis, that is the purpose of qualitative analysis.
Grounded theory
Grounded theory is a term often used loosely to describe qualitative analysis, as the categories and codes generated are ‘grounded’ in the collected data. However, it also refers to a precisely structured method consisting of set techniques (Reference Glaser and StraussGlaser & Strauss, 1967). Constant comparison of the data in four stages is designed to systematically generate coding categories until nothing new is revealed, i.e. the point of theoretical saturation is reached. These categories are then tested by theoretical sampling, that is, collection of further data on the basis of concepts shown in the earlier stages to be relevant. It is insufficient for a research paper merely to state that “data were analysed using grounded theory”: researchers must be explicit as to what is meant by this.
Observational data analysis
In observational studies, data are recorded in the form of field notes made by the observer at the time of the study. These field notes are then reviewed when the researcher is no longer in the ‘field’ (the observational situation) and written up as an ethnography. This combines a description of the setting with a theoretical framework for understanding events through the personal experience of the researcher.
Interview data analysis
Interviews, both individual and group, are transcribed to produce texts that can be used to generate coding categories and to test theories. This process is often described as content analysis, and can involve enumerating procedures such as counting word frequencies, sometimes aided by computer software (see below). The transcripts can be a simple record of the words spoken or more complex documents including annotations for pauses, intonation and other non-verbal expressions. Conversation analysis involves the combined use of both audio and visual recordings to study interactional practices in particular settings (Reference Atkinson and HeritageAtkinson & Heritage, 1984). Narrative analysis centres on personal accounts of experience or events, either via an oral history or written autobiographical documents.
Documentary data analysis
Content analysis can be used for documents in much the same way as it is for interview transcripts, although again an interpretative analysis that takes into account meaning and context is preferable to the quantitative word-count approach.
A third method of content analysis commonly applied to documents is discourse analysis. Influenced by the critical theorists such as Foucault, the emphasis is on the use of official documents by different social groups in order to regulate the actions of another (Reference Lupton and ChapmanLupton & Chapman, 1995). Thus, individual documents are interpreted in the light of their historical context and the pre-existing social relationships at that time.
Data management
Qualitative studies produce large amounts of data that can be difficult to manage, particularly when the data are being constantly reviewed and rearranged or recoded. Computer packages are available to assist with data handling, but it may take considerable time to input data from its raw field-note form. Coding programs within such packages can be useful, especially in facilitating team working on multi-site projects, but they may lead to an overemphasis on text-based analysis (Reference Weaver and AtkinsonWeaver & Atkinson, 1994). Problems include reliance on quantitative content analysis such as counting word frequencies in texts, which pays no attention to the context or meaning of words and can lead to false assumptions.
Evaluating the results of qualitative research
The criteria of relevance and validity must be met by both qualitative and quantitative research. Commissioners of research expect to find real answers to questions that matter. If qualitative research cannot meet these criteria then it will be of little value. When critically appraising a study that employs qualitative methods a number of questions are useful. These are summarised in Box 3 and discussed below. A good example of clarity of presentation in a study using qualitative methods is found in Donovan & Blake (2000).
Is the study relevant?
As with all research, publications presenting results should include a clear statement of the research question, justification for the chosen methodology and consideration of the clinical implications of results. It makes sense to consider both qualitative and quantitative methods in relation to the question before deciding which to use and stating why.
Are the results valid?
The term validity does not refer to absolute undisputed truths, which is how it is often used in research papers, but to a sound and well-reasoned argument. Thus, theoretical positions that tend to extremes – either that there is one and only one right answer (radical positivism) or that there are never any right answers (radical relativism) – both side-step validity issues. By holding an extreme position, they avoid the necessary explanation of reasoning – either “I must be right” or “There is no such thing as right.”
The way to demonstrate validity is in the presentation of methodology and results so that the reader can judge the points made. For this, clarity is essential. Specialist jargon should be avoided, although some qualitative researchers have been guilty of using terminology that is not accessible to the general reader. For example, the discovery of three new ‘types’ of validity (testimonial, catalytic and reflexive) is not particularly helpful (Reference StilesStiles, 1999). The proliferation of checklist guides on evaluating qualitative research are of no value if the terminology is inaccessible to health service researchers (Reference Chapple and RogersChapple & Rogers, 1998).
Qualitative research, like quantitative, has to demonstrate that data have been collected and analysed in a systematic way. The best way to do this is to be explicit about exactly what has been done. It also needs to be inclusive, considering all points of view or possibilities in the subject area, and not be biased towards one opinion or the other. Any results found that do not fit with the theories generated must be examined.
Reflexivity must also be considered. This is, arguably, a specialist term, although its use is no longer confined to disciplines specialising in qualitative methods. It means that researchers must take into account the effect of themselves on a study, both in what they bring to the design in terms of pre-existing theoretical positions and how their presence affects the research process. For example, Freud failed to consider how some of the assumptions in his theories were based on his position as a professional male in Viennese society, and so his theoretical generalisations were open to criticism by feminist writers (Reference MitchellMitchell, 1974).
Evaluating results of research that uses qualitative methods
Is the study relevant?
Is the research question clearly stated and is it a question worth answering?
What are the clinical implications of the results?
Why have qualitative methods been chosen instead of quantitative?
Are the results valid?
Are the authors explicit in what their methods entailed?
Have the data been collected systematically?
Have all possible perspectives on the data been included in the analysis?
Is the research inclusive: are exceptions to the theory generated discussed?
Have the authors taken into account reflexivity, i.e. the effect of their presence and the research process on the data collected?
Qualitative research in psychiatry?
This paper has demonstrated that there is nothing new or foreign in qualitative research methods for psychiatrists. There is no need for psychiatrists to be any more hostile or enthusiastic towards these techniques than we are to the more familiar methods currently published in our journals. There are certainly advantages in the judicious combination of qualitative and quantitative methods in all types of health service research (Reference BarbourBarbour, 1999). For the evidence base of psychiatry to move forward, we need to employ a range of methodologies carefully selected to fit the questions posed and to gather relevant data. As the introductory quotation states, evidence can, and indeed can only, be gathered from clinical practice.
Further reading
A comprehensive and highly recommended account of the history and practice of qualitative methods is given by Murphy et al (1998). This is free to all health service employees and can be downloaded in PDF format from the Health Technology Assessment website (http://www.hta.nhsweb.nhs.uk) or obtained from The National Co-ordinating Centre for Health Technology Assessment, Mailpoint 728, Boldrewood, University of Southampton, Bassett Crescent East, Southampton SO16 7PX. Another useful volume is edited by Pope & Mays (2000), and it contains many of the British Medical Journal articles referenced below. Finally, Grbich (1999) has written a good practical guide to qualitative techniques in health service research.
Multiple choice questions
-
1. Theoretical sampling is based on:
-
a convenience
-
b developing a theory
-
c randomisation
-
d intuition
-
e emerging categories.
-
-
2. Qualitative methods are useful in:
-
a hypothesis generation
-
b treatment evaluation
-
c investigating anomalous results
-
d exploring barriers to policy implementation
-
e finding out user views.
-
-
3. Qualitative methods include:
-
a census survey
-
b focus groups
-
c participant observation
-
d case–control study
-
e semi-structured interviews.
-
-
4. To demonstrate validity a qualitative study should be:
-
a explicit
-
b esoteric
-
c systematic
-
d exclusive
-
e reflexive.
-
-
5. Reflexivity involves:
-
a consideration of the impact of the presence of the researcher on data collected
-
b awareness of the researcher as a positioned observer
-
c achieving the lotus position
-
d recognition of the researcher's preconceptions and their influence on interpretation and analysis
-
e knowledge of the works of Karl Marx.
-
MCQ answers
| 1 | 2 | 3 | 4 | 5 | |||||
|---|---|---|---|---|---|---|---|---|---|
| a | F | a | T | a | F | a | T | a | T |
| b | T | b | F | b | T | b | F | b | T |
| c | F | c | T | c | T | c | T | c | F |
| d | F | d | T | d | F | d | F | d | T |
| e | T | e | T | e | T | e | T | e | F |
eLetters
No eLetters have been published for this article.