1. Introduction
Survey data can measure key aspects of the smoking cessation process, including the frequency and duration of quit attempts and the popularity and relative effectiveness of available cessation aids. For example, population surveys indicate that ~40% of smokers report having made at least one quit attempt in any given year [Reference Borland, Partos, Yong, Cummings and Hyland1]. The majority of quit attempts are unaided [Reference Edwards, Bondy, Callaghan and Mann2]; however, smokers who report using any type of cessation aid are more frequently using pharmacological support or e-cigarettes compared with, for example, face-to-face behavioural support or digital aids [Reference Beard, Brown, Michie, Kaner, Meier and West3, Reference Perski, Jackson, Garnett, West and Brown4]. The utility of survey data largely depends on the accuracy of respondents’ quit attempt histories. However, a nontrivial level of misreporting of quit attempt histories likely occurs. Such misreporting is particularly relevant for unsuccessful quit attempts (as people tend to remember successful attempts, e.g., through identifying as ex-smokers), which can have severe consequences for the estimation of the effectiveness of different quit aids and tobacco control policies [Reference Borland, Partos and Cummings5].
Memory processes such as forgetting are relevant to the misreporting of unsuccessful quit attempts in general and unsuccessful attempts involving specific cessation aids in particular [Reference Schacter6]. First, forgetting may simply occur due to decreasing accessibility of stored information over time. For example, Berg and colleagues used data from the English Smoking Toolkit Study to assess whether smokers’ estimated failure to report quit attempts varied as a function of the duration of the quit attempt and time since the quit attempt started. They estimated a strong trend for quit attempts that lasted for shorter periods and that started a longer time ago to fail to be reported [Reference Berg, An and Kirch7]. Second, inattentive or shallow processing of information may lead to the weak encoding of memories. A recent study in England found that reports on the use of digital aids in a smoking cessation attempt was low at 2.7% [Reference Perski, Jackson, Garnett, West and Brown4]. Smokers may fail to report unsuccessful attempts involving digital aids at a higher rate than attempts involving pharmacological support or contact with a healthcare professional, as the use of digital aids is typically discontinued during the first week of download [8]. Moreover, better recall of failed quit attempts among smokers using stop smoking medications compared with self-quitters has been observed [Reference Borland, Partos and Cummings5], which can lead to the underestimation of the effectiveness of certain quit aids. Third, current beliefs and retrospective distortions may influence the encoding of memories. For example, the belief that a cessation aid was not personally relevant or useful for quitting may lead to forgetting [Reference Perski, Blandford, Ubhi, West and Michie9]. Fourth, smokers may also reinterpret quit attempts that failed more quickly as not being “real attempts” [Reference Berg, An and Kirch7].
Using data from the English Smoking Toolkit Study, we aimed to extend previous findings by examining whether the estimated failure to report unsuccessful quit attempts varies by the type of cessation aid used.
2. Methods
2.1. Study Design and Setting
The study protocol and analysis plan were preregistered on the Open Science Framework (osf.io/k6q3d). This was a correlational study involving cross-sectional survey data. The STROBE guidelines were used in the design and reporting of the study [Reference Elm, Altman, Egger, Pocock, Gøtzsche and Vandenbroucke10]. The study is part of the ongoing Smoking Toolkit Study (STS), which involves monthly, face-to-face, computer-assisted household surveys with adults aged 16+ in England [Reference Fidler, Shahab and West11]. The STS uses a hybrid of random probability and quota sampling, which results in a sample that is representative of the adult population of smokers in England. Interviews are held with one household member. Informed consent is obtained prior to each interview. Ethical approval was granted by UCL’s Research Ethics Committee (2808/005).
2.2. Study Population
Data were collected from respondents surveyed between January 2014 (the first wave at which the use of e-cigarettes had stabilised—selected as starting point to reduce potential bias introduced by the increased popularity of e-cigarettes) and December 2020 (the latest wave of data available at the time of analysis). Respondents were included in the analyses if they were aged 16+ years, a current smoker, had made at least one serious quit attempt in the past year, and had complete data on the demographic and smoking variables of interest. This deviated from the preregistered study protocol, in which we had specified that recent ex-smokers would also be included. However, by definition, they cannot have forgotten their most recent quit attempt.
2.3. Measures
Respondents were asked to provide data on sex; age; social grade, measured with the British National Readership Survey’s Social Grade Classification Tool [12]; cigarettes per day and time to first cigarette [Reference Kozlowski, Porter, Orleans, Pope and Heatherton13]; and the number of serious quit attempts made in the past year (defined as deciding to try to never smoke again).
Smokers who made at least one serious quit attempt in the past year were asked to select which of the following cessation aids were used in their most recent quit attempt: (1) prescription nicotine replacement therapy (NRT), (2) NRT bought over the counter (OTC), (3) varenicline, (4) bupropion, (5) e-cigarettes, (6) face-to-face behavioural support, (7) telephone support, (8) written self-help materials, (9) digital support (i.e., websites/smartphone apps), (10) hypnotherapy, and (11) none of the above (“unaided”). Respondents were asked to indicate when their most recent quit attempt started, with response options including: (1) 1-7 days ago, (2) 8-30 days ago, (3) 31-60 days ago, (4) 61-90 days ago, (5) 91 days to 6 months ago, (6) >6 months to 1 year ago, and (7) don't know/not stated. Those reporting “don’t know/not stated” were excluded. Finally, respondents were asked how long their most recent quit attempt lasted, with response options including (1) still not smoking, (2) <1 day, (3) 1-7 days, (4) 8-30 days, (5) 31-60 days, (6) 61-90 days, (7) 91 days to 6 months, (8) >6 months to 1 year, and (9) don't know/not stated. Those reporting “still not smoking” or “don't know/not stated” were excluded. Responses that fell outside the realms of possibility (e.g., respondents indicating that their quit attempt started 1-7 days ago and lasted >6 months to 1 year) were also excluded. This had not been specified in the preregistered analysis plan.
2.4. Data Analysis
Data were analysed in RStudio v.1.2.5. The percentage “failure to report” was estimated for each cessation aid with an established analytic approach [Reference Berg, An and Kirch7]. We first standardised each of the temporal assessment periods to reflect the number of quit attempts at the population-level (and not per individual) that would be reported if all periods were one month long. For example, the number of quit attempts that started in the last week was multiplied by 4 to reflect the number of quit attempts that would be expected to occur over a 1-month period at the same rate. Longer time periods were divided appropriately.
The estimated “failure to report” was then derived by calculating the percentage of quit attempts of different lengths that failed to be reported for each time period, assuming that the rate of reporting for attempts that started most recently (e.g., in the last week) is most accurate and that the rate of quit attempts over time is uniform. For example, the percentage failure to report for quit attempts that started 8-30 days ago and lasted for 1-7 days was derived by dividing the standardised number of quit attempts by the number of quit attempts that started in the last week and lasted for 1-7 days.
Matrices of time since the quit attempts started by length of the quit attempts with percentages estimated failure to report and 95% bootstrap percentile confidence intervals (CIs) were produced and plotted for each cessation aid. We performed 1000 bootstrap replications, with the 2.5th and 97.5th percentiles of the empirical distribution forming the 95% bootstrap percentile CIs [Reference Efron14]. Plots were visually inspected to examine whether the estimated forgetting curves were differentially shaped for any of the cessation aids. In addition, tests for equality of proportions were performed with the prop.test function to examine whether quit attempts lasting between one day and one week and that started >6 months ago failed to be reported at a different rate depending on the cessation aid used. As previous research had suggested that smokers tend to fail to report unsuccessful quit attempts that lasted a short time and occurred a longer time ago [Reference Berg, An and Kirch7], we reasoned that if there is any moderation by cessation aid, it would be important to detect this for attempts that tend to be forgotten at a high (rather than low) rate. This had not been specified in the preregistered analysis plan.
As relapse curves differ by cessation aid (i.e., some aids are more effective) [Reference Jackson, Kotz, West and Brown15] [Reference Jackson, McGowan and Ubhi16–Reference Kotz, Brown and West18], we considered applying an “effectiveness adjustment” to the raw quit attempt numbers using estimates from [Reference Jackson, Kotz, West and Brown15, Reference Jackson, McGowan and Ubhi16]. However, as the effectiveness of each aid is already considered in the original analytic approach (with the ratio of expected vs. reported quit attempts estimated on the basis of smokers whose quit attempts lasted the same amount of time and “conferred” the same level of effectiveness at that moment), the effectiveness adjustment did not alter the percentages (see Supplementary File 1).
2.4.1. Planned Sensitivity Analyses
As there was an increasing trend in unaided quit attempts in England at the end of 2017 (http://www.smokinginengland.info/), we conducted a planned sensitivity analysis (SA) for unaided quit attempts using data from 2014 to the end of 2017 (see Supplementary File 2).
2.4.2. Unplanned Sensitivity Analyses
We also conducted two unplanned SAs, examining the percentage estimated failure to report when combining all respondents who used any pharmacological aid or any behavioural aid (excluding those who used both a pharmacological and a behavioural aid; “multiple aids”) to increase sample sizes (see Supplementary File 2).
3. Results
A total of 6,614 smokers who had made at least one serious quit attempt in the past year were surveyed, of whom 70 respondents had missing data on any of the demographic or smoking characteristics of interest, with a further 652 respondents with out of range or implausible combinations of started/lasted values, yielding a total sample of 5,892 (89.1%) respondents with complete data on all variables of interest. The majority of unsuccessful quit attempts lasted less than one month (see Table 1).
NRT: nicotine replacement therapy; OTC: over the counter.
* As cessation aids were not mutually exclusive (i.e., respondents could select multiple options), the total percentage exceeds 100%.
The matrix of time since the quit attempt started by length of the quit attempt and percentages estimated failure to report for unaided attempts is presented in Table 2 and Figure 1. Supplementary File 2 illustrates the rate of estimated failure to report for each cessation aid. We were unable to estimate the rate of failure to report for attempts involving bupropion, telephone support, written support, and hypnotherapy due to small sample sizes (i.e., >3 cells with 0 reported quit attempts). In addition, bootstrap CIs could not be estimated for percentages equal to 0 or 100. Overall, we estimate that a substantial proportion of unsuccessful quit attempts fail to be reported. This failure is particularly prominent for attempts that last a short time or occurred longer ago. Tests for equality of proportions indicated that, compared with unaided attempts, the percentage failure to report quit attempts that lasted between one day and one week and that started >6 months ago was significantly lower for attempts involving behavioural support (92% vs. 75%,
, p = 0.002). No other significant differences were detected (all p’s > 0.05; see Table 3).
Raw n: the number of respondents indicating an unsuccessful quit attempt; Standardised n: the estimated number of unsuccessful quit attempts based on our calculations.
Percentage estimated failure to report quit attempts of varying lengths (indicated by the line colour) and varying times since the quit attempt started (x-axis).
4. Discussion
In smokers in England, we estimate that a substantial proportion of unsuccessful quit attempts fail to be reported. This failure appears particularly prominent for attempts that last a short time or occurred a long time ago. Compared with unaided attempts, the estimated percentage failure to report quit attempts that lasted between one day and one week and that started >6 months ago was significantly lower for attempts involving face-to-face behavioural support. Our results replicate those reported by Berg and colleagues a decade ago [Reference Berg, An and Kirch7] and suggest that smokers may have somewhat poorer memory of unaided quit attempts compared with attempts involving behavioural support. A potential explanation for the estimated improved memory of unsuccessful attempts involving face-to-face behavioural support (compared with unaided attempts) may be due to such support involving a series of activities with emotional and cognitive salience (e.g., transportation to face-to-face meetings and conversations with and accountability to a healthcare professional).
Borland and colleagues have previously discussed how differential failure to report unsuccessful quit attempts may have consequences for the estimation of the effectiveness of treatments [Reference Borland, Partos and Cummings5]. For example, we found that people appeared more likely to forget unsuccessful unaided attempts compared with those involving behavioural support. Insofar that this generalises, studies using retrospective surveys to estimate the comparative effectiveness of behavioural support will underestimate its effectiveness. Although clinical guidelines for smoking cessation are primarily underpinned by evidence from randomised controlled trials (which are not subject to differential failure to report), policy evaluations and related decisions sometimes rely on retrospective/cross-sectional survey data. This may lead to overestimations of policy effects, as the failure to report quit attempts that occurred a longer time ago (i.e., prior to the implementation of the new policy) contributes to the comparatively lower quit rates reported in the pre- compared with the postintervention period [Reference Borland, Partos and Cummings5]. Therefore, the estimated failure to report quit attempts in the present study adds to the existing literature indicating that this is a serious issue for retrospective/cross-sectional survey data.
4.1. Limitations
First, respondents were only asked about “serious” quit attempts. However, it is plausible that a differently worded question may have captured a larger number of attempts, including those that (i) were “serious” at the outset but were retrospectively reclassified as less serious after having failed relatively quickly and (ii) were not regarded as “serious” by smokers at the outset but evolved into a sustained attempt to stop [Reference Hughes and Callas19, Reference Hughes, Solomon, Naud, Fingar, Helzer and Callas20]. Second, the popularity and effectiveness of different aids are likely to have impacted the results. We considered applying an “effectiveness adjustment” to the raw numbers but concluded that this would not alter the percentage estimated failure to report due to the analytic approach. Third, the results are dependent on the validity of the approach used to estimate the percentage failure to report quit attempts (i.e., the assumption that the rate of reporting for attempts that started in the last week is most accurate and that the rate of population-level quit attempts over time is uniform). For example, the estimate that 76% of those making an unaided quit attempt that lasted 2-3 months and that started 6-12 months ago would fail to report their attempt appears higher than expected and is substantially higher than the corresponding estimate (i.e., 8.5%) reported by Berg and colleagues [Reference Berg, An and Kirch7]. It is likely that the estimates in the present study were sensitive to the small sample sizes for many of the quit aids, thus lacking precision, which limits strong conclusions. As it is difficult to test the validity of our method (i.e., there is no “gold standard” method for comparison), we recommend using triangulation across multiple methods and data sources to arrive at more precise forgetting estimates. However, it should be noted that the key assumption underpinning our method—i.e., that serious quit attempts that started in the last week should be accurately reported (which is grounded in decades of memory research)—supports its validity [Reference Schacter6]. Fourth, as the cessation aids were not mutually exclusive (i.e., participants could indicate multiple options), this may have limited the ability to detect differences in the estimated percentage failure to report for the different cessation aids. Finally, our sample was young, the majority were light smokers, and due to low cell counts, we were unable to estimate the percentage failure to report for quit attempts involving bupropion, telephone support, written support, and hypnotherapy, which likely limits the generalisability of the results.
4.2. Implications and Future Directions
The overall finding that a large proportion of unsuccessful quit attempts may fail to be reported has implications for the assessment of quit attempt histories. Public health researchers should consider triangulating survey data with qualitative methods (e.g., following up smokers a period after their quit attempt) and ecological momentary assessments (i.e., brief, regular surveys delivered in or near real-time on people’s mobile phones) [Reference Stone and Shiffman21], as this may help elucidate why smokers have poor memory of unsuccessful quit attempts. The finding that the estimated failure to report appeared lower for attempts involving face-to-face behavioural support compared with unaided attempts may be interpreted to suggest that interactions with stop smoking counsellors lead to deep information processing and hence strong encoding of memories [Reference Schacter6]; however, this would need to be corroborated in future research.
5. Conclusion
In smokers in England, a substantial proportion of unsuccessful quit attempts may fail to be reported. This failure appears particularly prominent for attempts that last a short time or occurred longer ago and appears lower for attempts involving behavioural support compared with unaided attempts.
Data Availability
The data underpinning the findings of this study are available on request from the senior author, JB.
Disclosure
The funder had no final role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication. All researchers listed as authors are independent from the funders, and all final decisions about the research were taken by the investigators and were unrestricted. The views expressed are those of the authors and not necessarily those of the funders.
Conflicts of Interest
OP has no conflicts of interest to declare. RW undertakes research and consultancy for and receives travel funds and hospitality from manufacturers of smoking cessation medications (Pfizer, GlaxoSmithKline, Johnson & Johnson). JB has received unrestricted research funding from Pfizer to study smoking cessation.
Acknowledgments
We gratefully acknowledge all funding listed. The authors would like to thank David Simons for his help with the R code. This work was supported by the Cancer Research UK (C1417/A22962). The UK Department of Health, Pfizer, GlaxoSmithKline, and Johnson and Johnson have also all contributed funding to the data collection for the Smoking Toolkit Study.
Supplementary Materials
Supplementary 1. Supplementary File 1: matrix of time since the quit attempt started by the length of the quit attempt and percentages estimated failure to report for unaided attempts, without and with an “effectiveness adjustment.”
Supplementary 2. Supplementary File 2: percentage estimated failure to report quit attempts of varying lengths (indicated by the line colour) and varying times since the quit attempt started (x-axis) for each cessation aid, any pharmacological aid, any behavioural aid, or unaided attempts with data up to 2017 (sensitivity analyses).
Open access
