Loneliness is the discrepancy between people's aspirations for social relationships and the reality of these relationships (Cacioppo & Patrick, Reference Cacioppo and Patrick2008; Weiss, Reference Weiss1973). Loneliness is now widely recognized as a public health priority given its various detrimental consequences on physical and mental health (Ding, Eres, & Surkalim, Reference Ding, Eres and Surkalim2022; Lee et al., Reference Lee, Pearce, Ajnakina, Johnson, Lewis, Mann and Lewis2021a). Interestingly, because many large epidemiological studies have investigated the prevalence and consequences of loneliness (e.g. Surkalim et al., Reference Surkalim, Luo, Eres, Gebel, van Buskirk, Bauman and Ding2022), the use of very short measures of this construct is expanding (Maes, Qualter, Lodder, & Mund, Reference Maes, Qualter, Lodder and Mund2022). Indeed, a limitation of epidemiological studies lies in their large number of variables to collect, which can make it difficult to include lengthy measures for a given construct. Therefore, because standard loneliness scales typically have a relatively large number of items, they are often shortened to just a few items for the sake of feasibility (Hughes, Waite, Hawkley, & Cacioppo, Reference Hughes, Waite, Hawkley and Cacioppo2004). Although these shortened versions appear to have adequate psychometric properties, few studies have yet directly compared the validity of these scales, or have yet compared the estimated prevalence of loneliness when using standard and shortened scales (Lin et al., Reference Lin, Tsai, Fan, Griffiths, Chang, Yen and Pakpour2022). It thus remains important to further characterize the consequences of using very short scales when measuring loneliness.
Various questionnaires have been developed over the years to measure loneliness (Maes et al., Reference Maes, Qualter, Lodder and Mund2022). One of the most widely used measures is the UCLA Loneliness Scale (Russell, Reference Russell1996). This 20-item questionnaire (UCLA-LS-20) was originally developed to provide a unidimensional loneliness measure in young adults. Numerous studies have examined the convergent and discriminant validity of this scale, as well as its internal and test–retest reliability (Alsubheen, Oliveira, Habash, Goldstein, & Brooks, Reference Alsubheen, Oliveira, Habash, Goldstein and Brooks2021; Cole, Bond, Qualter, & Maes, Reference Cole, Bond, Qualter and Maes2021). The relatively good psychometric qualities of this instrument have contributed to its massive dissemination and adoption by many researchers interested in loneliness.
Numerous epidemiological studies have examined the prevalence and risk factors for loneliness in recent decades (e.g. Lasgaard, Friis, and Shevlin, Reference Lasgaard, Friis and Shevlin2016). One obstacle to the use of the UCLA-LS-20 in epidemiological studies has been its large number of items, which has created feasibility problems. Consequently, a shorter version of the UCLA-LS-20 with only three items (UCLA-LS-3) was developed in the early 2000s (Hughes et al., Reference Hughes, Waite, Hawkley and Cacioppo2004). Shortly after its creation, a large number of studies focused on the psychometric properties of the UCLA-LS-3 (e.g. Trucharte et al., Reference Trucharte, Calderón, Cerezo, Contreras, Peinado and Valiente2023). As with the original version, relatively good psychometric properties were reported. However, very few studies have made direct comparisons (i.e. head-to-head comparisons) between the UCLA-LS-20 and the UCLA-LS-3. These direct comparisons are of particular importance as they permit the clear identification of the measurement differences that result from the choice of a short measure v. a long measure. Indeed, indirect comparisons (i.e. comparing the psychometric properties of the UCLA-LS-20 and the UCLA-LS-3, assessed in separate studies), are always subject to biases related to sample characteristics and, more generally, to methodological differences between studies that may affect the results found. It is therefore essential to conduct direct comparisons between the UCLA-LS-3 and UCLA-LS-20 in order to gain a more comprehensive understanding of the discrepancies in psychometric properties between the two scales.
Such head-to-head comparisons have been recently performed in a general population (Mund et al., Reference Mund, Maes, Drewke, Gutzeit, Jaki and Qualter2023), By comparing the nomological net of the UCLA-LS-3 and UCLA-LS-20 with a large number of variables, the authors found that the two scales present a similar nomological profile overall. However, although the correlational analyses used allowed for the comparison of the magnitude of associations with different variables, they do not indicate whether the two scales explain a common or unique pattern of variance for each variable. Moreover, this study did not assess the degree of agreement between the scales when the loneliness construct was dichotomized, a common practice in the literature. Indeed, even if the construct of loneliness is inherently dimensional, and even if the categorization of dimensional variables is typically not recommended from a statistical standpoint (Bennette & Vickers, Reference Bennette and Vickers2012), authors nevertheless frequently dichotomize this construct. The categorization of dimensional variables can be done, for example, to facilitate the interpretation of results for stakeholders (e.g. Heimke et al., Reference Heimke, Furukawa, Siafis, Johnston, Engel, Furukawa and Leucht2024), or to identify subgroups that should benefit from a prevention or healthcare strategy. Therefore, we contend that a direct comparison of the dichotomized versions of the UCLA-LS-20 and UCLA-LS-3 is essential.
The overarching aim of this study was to gain a better understanding of the consequences of reducing the number of items in measures of loneliness with regard to their psychometric properties. Specifically, we conducted a head-to-head comparison of the UCLA-LS-20 and its shortened version (UCLA-LS-3) in terms of internal reliability, convergent validity (using mental health variables), discriminant validity (using demographical variables), and known-groups validity (using marital status). Furthermore, we also compared the prevalence estimates of loneliness obtained from both scales, and we explored the sensibility and specificity of the UCLA-LS-3 in comparison to the UCLA-LS-20. Because the measures of loneliness are affected by issues of measurement invariance across age groups (Panayiotou, Badcock, Lim, Banissy, & Qualter, Reference Panayiotou, Badcock, Lim, Banissy and Qualter2023), we conducted these comparisons in a homogeneous sample of young adults.
Methods
Open science
In accordance with the Ethical Committee, the processed data are available upon request only. The post-hoc nature of this study prevented us from pregistering our analysis plan, but the R code used to analyze the data is publicly shared at https://github.com/CorentinJGosling/GOSLING_UCLA_LS.
Recruitment process
The study was conducted between November 2, 2023 and December 11, 2023. All midwife students in their second to fifth year of study were invited to participate in the anonymous online survey. The invitation was made by email (institutional email address), and all the students from the 34 faculties of France were contacted. To contact the students, we intended to send an invitation email once a week for 4 weeks. However, because some faculties did not send at least one email after 2 weeks, the national students’ association also put a weblink to the survey on social media (Twitter, Facebook, Instagram).
Ethics
We ensured participants’ information and obtained informed consent from all participants before inclusion regarding the different approved studies through a transparency portal following the [anonymized] Data Protection ([anonymized]). [anonymized] ethics committee approved the project ([anonymized]). The study was furthermore registered to the [anonymized].
Participants
According to public data (https://drees.solidarites-sante.gouv.fr/sources-outils-et-enquetes/lenquete-annuelle-sur-les-ecoles-de-formation-aux-professions-de-sante), about 3700 French midwifery students were eligible for the survey. Among them, 2063 started to answer the survey, and 1742 (86%) had no missing data at the key variables required for the present study, and were thus included in final analyses (see the pattern of missingness in Supplementary Results S1).
Measures
UCLA Loneliness scale (20-points)
The 20-item UCLA Loneliness Scale (UCLA-LS-20) is a self-report questionnaire measuring loneliness. The possible responses ranged from ‘not at all’ (1) to ‘often’ (4). The total score thus ranges from 20 to 80, with a higher score reflecting higher loneliness (de Grâce, Joshi, & Pelletier, Reference de Grâce, Joshi and Pelletier1993; Russell, Peplau, & Ferguson, Reference Russell, Peplau and Ferguson1978). The most common cut-off value used to categorize the scores obtained at the UCLA-LS-20 is probably the score >43 (Lee et al., Reference Lee, Cho, Yang, Chang, Ryu, Noh and Park2021b). However, because other cut-offs have been used (Surkalim et al., Reference Surkalim, Luo, Eres, Gebel, van Buskirk, Bauman and Ding2022), we also conducted sensitivity analyses with other cut-off values to assess the robustness of our analyses dichotomizing the UCLA-LS-20.
UCLA Loneliness scale (3-points)
The 3-item UCLA Loneliness Scale (UCLA-LS-3) is a shortened version of the UCLA-LS-20 that contains only three items, rated on a scale that ranges from ‘not at all’ (1) to ‘often’ (3) (Hughes et al., Reference Hughes, Waite, Hawkley and Cacioppo2004). The total score thus ranges from 3 to 9, with a higher score reflecting higher loneliness. The UCLA-LS-3 was selected over other shorter versions of the UCLA-LS-20 (e.g. the UCLA-LS-6 or UCLA-LS-8) due to its extensive usage, which confirms the suitability of this limited number of items in addressing the needs of researchers. Many different cut-off values have been used to categorize the scores obtained at the UCLA-LS-3 (Surkalim et al., Reference Surkalim, Luo, Eres, Gebel, van Buskirk, Bauman and Ding2022).
PHQ-9
The PHQ-9 is a self-report questionnaire measuring depression. The nine items explore the severity and frequency of depressive symptoms and are associated with a 4-point Likert-type scale ranging from 0 (not at all) to 3 (nearly every day) (Kroenke & Spitzer, Reference Kroenke and Spitzer2002). The total score ranges from 0 to 27, with a higher score reflecting higher depressive symptoms. We assessed the convergent validity of the UCLA-LS-20 and UCLA-LS-3 by exploring their association with the PHQ-9 measure, as loneliness is known to be related to depression (Weeks, Michela, Peplau, & Bragg, Reference Weeks, Michela, Peplau and Bragg1980).
GAD-7
The GAD-7 is a self-report questionnaire measuring anxiety. The seven items explore the severity and frequency of generalized anxiety disorder symptoms and are associated with a 4-point Likert-type scale ranging from 0 (not at all) to 3 (nearly every day) (Kroenke, Spitzer, Williams, Monahan, & Löwe, Reference Kroenke, Spitzer, Williams, Monahan and Löwe2007). The total score ranges from 0 to 21, with a higher score reflecting higher anxiety symptoms. We assessed the convergent validity of the UCLA-LS-20 and UCLA-LS-3 by exploring their association with the PHQ-9 measure, as loneliness is known to be related to anxiety (Santini et al., Reference Santini, Jose, York Cornwell, Koyanagi, Nielsen, Hinrichsen and Koushede2020).
Marital status
The marital status was a self-reported dichotomous variable (‘Single’ v. ‘In a relationship’). We explored the known-groups validity by comparing the loneliness values of the two scales between participants that were currently in a relationship to those that were single, as this variable is known to be a strong predictor of loneliness (Page & Cole, Reference Page and Cole1991).
Age
The age of participants was self-report in an un-identifying ordinal scale (‘18–19’, ‘20–21’, ‘22–23’, ‘24–25’, and ‘>25’). This age scale was established in accordance with the guidelines set forth by our Ethical Committee to ensure the complete anonymity of the participants. We explored the discriminant validity of the two scales by exploring their association with age, since several studies highlighted a measurement invariance issue of loneliness measures with age (e.g. Panayiotou et al., Reference Panayiotou, Badcock, Lim, Banissy and Qualter2023). As our sample of students has limited age variability, this exploration was designed to ensure that age does not influence the loneliness scores within the population for which the UCLA-LS was originally developed (students).
Financial difficulties
Financial difficulties were measured on a self-reported ordinal scale ranging from ‘No difficulties’ to ‘Very important’. Discriminant validity was explored by looking at the association with financial difficulties, as the construct of loneliness is theoretically not intended to heavily depend on financial difficulties.
Data analysis
All statistical analyses were conducted in the R environment (version 4.1.1).
First, as preliminary analysis, we explored the association between the UCLA-LS-3 and UCLA-LS-20 using zero-order Pearson's correlation and disattenuated correlation (using the CTT R package, Willse, Reference Willse2018).
Second, for the psychometric properties, we started by estimating the internal reliability of the two scales (using Cronbach's alpha and McDonald's omega), and comparing their values (using the formulas described by Feldt (Reference Feldt1980) and using the cocron R package; Diedenhofen, Reference Diedenhofen2016). To explore the known-groups validity, we estimated the standardized mean difference of loneliness scores depending on the marital status using the metaConvert R package (Gosling et al., Reference Gosling, Cortese, Solmi, Haza, Vieta, Delorme and Raduaunder review). Then, for each assessment of the convergent/discriminant validity, we built a commonality analysis model with either the GAD-7, PHQ-9, age, or financial difficulties as outcome, and both the UCLA-LS-3 and UCLA-LS-20 as predictors (using the yhat R package, Nimon and Oswald, Reference Nimon and Oswald2013). From this model, we extracted the standard zero-order Pearson's correlation between the predictors and the outcome, the total percentage of variance explained by the UCLA-LS-20 and UCLA-LS-3, and their commonality and uniqueness coefficients (i.e. the percentage of variance of the outcome commonly/uniquely explained by the UCLA-LS-3 and UCLA-LS-20). We then estimated whether the percentage of variance uniquely explained by the UCLA-LS-20 was superior to that of the UCLA-LS-3, by running 10 000 bootstrap simulations for the 95% CI of the difference. In an alternative approach, we also ran disattenuated correlations between the UCLA-LS scales and the PHQ-9 and GAD-7, and we compared the magnitude of the correlation coefficients between the two scales and other variables. However, because this direct comparison of the magnitude of the (disatenuatted) correlations between the two loneliness scales and the PHQ-9, GAD-7, age and financial difficulties always led to similar conclusions as our commonality analyses, these results are only presented in the Supplementary Materials for parsimony.
Last, we compared the prevalence of loneliness determined by each scale, as well as the sensitivity and specificity of the UCLA-LS-3 against the UCLA-LS-20. To be able to explore the prevalence of the two scales, we first needed to dichotomize them. We chose the standard score of 43 as the cut-off for the UCLA-LS-20 in our main analyses. For the UCLA-LS-3, there is no consensus on the way to dichotomize it. We thus determined the optimal cut-off value (i.e. maximizing the sensitivity and specificity of the UCLA-LS-3 against the UCLA-LS-20) by using ROC curve analysis using the cutpointr R package (Thiele & Hirschfeld, Reference Thiele and Hirschfeld2021). Then, we estimated the prevalence of loneliness according to the two scales, we compared them using a McNemar test for paired proportions, and we assessed the sensitivity and specificity of the UCLA-LS-3 against the UCLA-LS-20. Last, as a robustness analysis, we replicated all these analyses but using other commonly employed cut-off scores for both the UCLA-LS-3 (a score ⩾6, ⩾7, or one of the items scored as ‘often’) and the UCLA-LS-20 (a score ⩾39 and ⩾53) (Surkalim et al., Reference Surkalim, Luo, Eres, Gebel, van Buskirk, Bauman and Ding2022).
Results
R code supporting data analysis, and a complete presentation of the results are presented in the Supplements S1–S7, available online (https://corentinjgosling.github.io/GOSLING_UCLA_LS/).
Description of the sample and preliminary analysis
The key demographic characteristics of the sample are presented in Table 1 and Supplementary Tables and Figures S1 and S2. Briefly, our sample was mainly composed of young women which was appropriately divided between the different years of study (from the 2nd to the 5th).
Demographic characteristics of the sample
Critically, we found a strong association between the UCLA-LS-20 and UCLA-LS-3 (Pearson's r = 0.675, 95% CI 0.649–0.700; disattenuated r = 0.781). This preliminary result, like a quality check of the data, confirmed that the two scales were measuring a similar construct in our sample.
Internal reliability
We found that both scales had an adequate internal reliability (α UCLA−LS−20 = 0.93, α UCLA−LS−3 = 0.80; ω UCLA−LS−3 = 0.81, Supplementary Text S3). The Feld's test revealed that the Cronbach's alpha for the UCLA-LS-20 was higher compared to the UCLA-LS-3 (p-value <0.001), which is not surprising given the reduced number of items and the reduced number of points in the scale of the UCLA-LS-3 (Cortina, Reference Cortina1993).
Known-groups validity
When comparing participants engaged in a relationship to those that were single, we found – for both scales – that single participants had higher loneliness scores (all p-values <0.05; Supplementary Table S4). Importantly, the associated effect sizes were very similar for the two scales (SMDUCLA−LS−20 = −0.130, 95% CI−0.226 to −0.035; SMDUCLA−LS−3 = −0.160, 95% CI −0.256 to −0.065).
Convergent and discriminant validity
We found that the UCLA-LS-20 and UCLA-LS-3 had a very similar profile when exploring their association with the PHQ-9 and GAD-7 (convergent validity) and the age and financial difficulties (discriminant validity) (see Figure 1 Supplementary Tables S5 and S6).
Percentage of common, total, and unique variance of the four outcomes explained by the UCLA-LS-3 and UCLA-LS-20.
For the convergent validity, our commonality analyses revealed that the UCLA-LS-3 and UCLA-LS-20 explained a significant proportion of the variance in the outcomes. Specifically, the two scales explained approximately 25% of the variance in the PHQ-9 and 20% of the variance in the GAD-7 (see Supplementary Tables S5). It is important to note that a significant proportion of this explained variance was shared by the UCLA-LS-3 and UCLA-LS-20 (21% of the variance of the PHQ-9 is shared by the two scales, and 14% of the variance of the GAD-7 is shared by the two scales). The UCLA-LS-3 did not uniquely explain a smaller (or larger) proportion of the variance in PHQ-9 and GAD-7 than UCLS-LS-20 (both p-values >0.05).
For the discriminant validity, our commonality analyses systematically revealed that the UCLA-LS-3 and UCLA-LS-20 both explained a small proportion of the variance of the age (<1%) and financial difficulties (<6%). A substantial part of this explained variance was again common to the two scales, and no scale explained more variance compared to the other (both p-values >0.05).
Prevalence
The ROC curve analyses revealed that, in our sample, a cut-off value ⩾6 or ⩾7 was generally optimizing the sensitivity and specificity of the UCLA-LS-3 (against the UCLA-LS-20). As shown in Fig. 2, the sensitivity and/or specificity of the UCLA-LS-3 were systematically below the expected threshold (80%). Moreover, the prevalence estimates of loneliness were often markedly different between the scales (Fig. 2). Critically, very slight variations in cut-off values for the UCLA-LS-3 (e.g. a 1-point increase, from ⩾6 to ⩾7), dramatically modified the prevalence estimated (45% v. 23%, respectively). All these sensitivity analyses are presented in detail in Supplementary Tables and Figures S7.
Sensitivity and specificity of the UCLA-LS-3 against the UCLA-LS-20, and prevalence of loneliness according to the two scales, and for various cut-off values.
Discussion
The present study conducted an in-depth, head-to-head assessment of the psychometric properties of two major scales for measuring loneliness. Our results revealed two major findings. First, when the scales are used dimensionally, their psychometric properties (internal reliability, validity) are very good and are of similar magnitude for both scales. Second, when the scales are dichotomized, some discrepancies between the scales were observed. Indeed, we found that the sensitivity and/or specificity of the UCLA-LS-3 against the UCLA-LS-20 were below acceptable threshold, regardless of the dichotomization process employed. In addition, we found substantial differences in the prevalence estimated by the UCLA-LS-3 – even with a minor change in cutoff (e.g. moving from a ⩾6 to a ⩾7 cutoff resulted in a decrease in prevalence of loneliness from 45% to 23%).
Our results generally confirmed, and extended, those from previous studies. As others, we have been able to demonstrate the adequate properties of the UCLA-LS-3 and UCLA-LS-20 when a dimensional scoring is used (Alsubheen et al., Reference Alsubheen, Oliveira, Habash, Goldstein and Brooks2021; Hughes et al., Reference Hughes, Waite, Hawkley and Cacioppo2004). However, our comparative analyses also systematically allowed us to demonstrate that the psychometric properties of the two scales were of similar magnitude. For example, our commonality analyses showed that a large part of variance in various outcomes (anxiety and depressive symptoms, age, and financial difficulties) was jointly explained by both scales. This result is directly in line with previous studies, that showed a moderate-to-strong association of loneliness with anxiety and depression (Pitanupong, Anantapong, & Aunjitsakul, Reference Pitanupong, Anantapong and Aunjitsakul2024; Trucharte et al. Reference Trucharte, Calderón, Cerezo, Contreras, Peinado and Valiente2023). Therefore, on top of confirming the good psychometric properties of both scales, our study demonstrated that the use of the UCLA-LS-3 for the dimensional measurement of loneliness did not result in a critical loss of information compared to the use of the UCLA-LS-20. This finding is particularly important for studies that require a short measure of loneliness, such as epidemiological studies.
However, critically, we showed that the concordance between the two scales is not as clear after a dichotomization. These analyses are particularly important because, although the loneliness construct is inherently dimensional, studies often dichotomize these scales, either to compare low/high loneliness participants on various criteria or to estimate loneliness prevalence. We found that the results about the concordance of the dichotomized versions of the two scales were highly variable depending on the way in which they were dichotomized. This result is all the more concerning as our study was limited to the use of dichotomization processes that had already been implemented by previous studies. For the UCLA-LS-3, the observed variability may be due, at least in part, to the reduction in the number of items compared with the UCLA-LS-20 and to the reduction in the response scale (which is 4 points in the UCLA-LS-20 and 3 points in the UCLA-LS-3). We believe that a very promising line for improving the UCLA-LS-3 categorization process would be to take up a larger response scale, as has been proposed, for example, by Klein et al. (Reference Klein, Zenger, Tibubos, Ernst, Reiner, Schmalbach and Beutel2021). For the UCLA-LS-20, we also found that prevalence estimates were variable depending on the cut-off used. This clearly calls for further studies to examine the specificity/sensitivity of the UCLA-LS-20 against a more refined assessment of loneliness, such as a combination of semi-structured interviews and observational measures.
A crucial methodological decision in the course of this study was the assumption that the scales utilized for measuring loneliness were indicative of a unidimensional construct. While (i) the UCLA-LS-20 was indeed developed with the assumption that it would yield results for a unidimensional construct (Russell, Peplau, & Cutrona Reference Russell, Peplau and Cutrona1980), and (ii) numerous studies have corroborated this factor structure (e.g. Dodeen, Reference Dodeen2015), other research has identified different factor structures (e.g. Cacioppo et al., Reference Cacioppo, Hawkley, Ernst, Burleson, Berntson, Nouriani and Spiegel2006). However, studies that identified more complex factor structures typically failed to agree on a common structure. Therefore, it seemed appropriate to retain the original structure that was chosen when developing the scale.
The present study should be interpreted in light of its limitations. First, it should be noted that our sample was composed of relatively young midwife students. While it has been common to assume measurement invariance in loneliness between the age groups, recent evidence casted doubts regarding this hypothesis (Panayiotou et al., Reference Panayiotou, Badcock, Lim, Banissy and Qualter2023). Therefore, while the low association of our scales with the age of our participants confirmed the soundness of our results in the present study, these results cannot be directly generalized to very different age groups (e.g. elderly people). Despite our large sample size, it is thus critical to conduct further comparative studies about the psychometric properties of the UCLA-LS-20 and UCLA-LS-3 in older populations. Second, while our sample was composed of WEIRD participants (White, Educated, Industrialized, Rich, Democratic; Henrich, Heine, and Norenzayan, Reference Henrich, Heine and Norenzayan2010), many data showed that loneliness was strongly related to culture. For example, an international survey analyzed the frequency of loneliness reported by dozens of thousands of participants aged 16–99 years, living across 237 countries, and found that loneliness was greater in individualistic cultures, and could interact with other variables such as gender (Barreto et al., Reference Barreto, Victor, Hammond, Eccles, Richins and Qualter2021). Replication of our results in more diverse samples is thus required to further enhance our understanding of the measurement invariance of loneliness. Third, because of potential differences in measurement error between the UCLA-LS-3 and UCLA-LS-20, it would have been interesting to use disattenuated correlations in the commonality models. However, to the best of our knowledge, this procedure has not yet been implemented in an R package.
Overall, the present study confirmed the psychometric properties of the UCLA-LS-3 and UCLA-LS-20 as dimensional measures of loneliness, and showed that that the use of the UCLA-LS-3 did not greatly modify the associations of observed loneliness levels with other key variables, such as mental health, age or some other demographic variables. These results confirm the relevance of the use of the UCLA-LS-3 in time-limited loneliness studies, at least in young samples of adults. On a more nuanced note, we have been able to show that the sensitivity and/or specificity of the UCLA-LS-3 against the UCLA-LS-20 was lower than what could have been expected, and that the choice of the dichotomization process greatly affected the prevalence estimates. Therefore, future studies categorizing loneliness measured via short scales should consider these results, and future meta-analyses exploring the prevalence of loneliness should combine data from studies using similar scales and scoring procedures (or at least should consider the impact of the variables on the prevalence estimates).
Data availability statement
The processed data that support the findings of this study are available upon reasonable request.
Acknowledgements
The authors acknowledge the French Association of Midwifery Students (ANESF) and thank the participants. They also sincerely thank Ms Angela Jackson who programmed the questionnaire on the REDCAP platform.
Competing interests
The authors declare that they have no relevant or material financial interests that relate to the research described in this paper.
Open access
