Hostname: page-component-7c8c6479df-r7xzm Total loading time: 0 Render date: 2024-03-28T11:54:07.379Z Has data issue: false hasContentIssue false

Short Assessment of Health Literacy (SAHL) in Portugal: development and validation of a self-administered tool

Published online by Cambridge University Press:  15 February 2018

Carla Pires*
Affiliation:
Researcher, Department of Social Pharmacy, Faculty of Pharmacy, University of Lisbon, Lisbon, Portugal
Pedro Rosa
Affiliation:
Auxiliary Professor, School of Psychology and Life Sciences, Lusophone University of Humanities and Technologies, Lisbon, Portugal Cognition and People-Centric Computing Laboratories (COPELABS), ULHT, Lisbon, Portugal Instituto Universitário de Lisboa (ISCTE-IUL), Cis-IUL, Lisbon, Portugal Grupo Internacional de Investigación Neuro-Conductual (GIINCO), Barranquilla, Colombia
Marina Vigário
Affiliation:
Associate Professor, School of Arts and Humanities, Centre of Linguistics of the University of Lisbon, University of Lisbon, Lisbon, Portugal
Afonso Cavaco
Affiliation:
Associate Professor, Department of Social Pharmacy, Faculty of Pharmacy, Research Institute for Medicines and Pharmaceutical Sciences, University of Lisbon, Lisbon, Portugal
*
Correspondence to: Carla Pires, Researcher, Department of Social Pharmacy, Faculty of Pharmacy, University of Lisbon, Avenida Professor Gama Pinto, 1649-003 Lisbon, Portugal. Email: cmbpires@gmail.com
Rights & Permissions [Opens in a new window]

Abstract

The goal of this study was to adapt, improve and validate a short, self-administered health literacy assessment tool for European Portuguese-speaking adults. Health literacy tools are of great importance to health authorities and professionals, as low or inadequate health literacy, that is, a limited capacity to handle health-related information, is associated with higher morbidity and mortality. The 18-item Short Assessment of Health Literacy for Brazilian Portuguese-speaking adults (SAHLPA-18) was adapted into European Portuguese. The European Portuguese tool (SAHLPA-23) includes five additional items. The SAHLPA-23 was tested in a convenience sample of 503 participants from two Portuguese regions. Socio-demographic data, literacy and cognitive indicators were collected. Participants also completed a questionnaire on comprehension of written health materials. Construct validity was assessed through correlations between SAHLPA-23 scores and education, literacy, and cognitive variables and score on the comprehension questionnaire. The psychometric properties of the new tool were compared with those of the SAHLPA-18. The mean SAHLPA-18 and SAHLPA-23 scores were 13.9 (77.2%; SD=2.9) and 18.3 (79.6%; SD=3.8), respectively. Both tools showed adequate reliability (Cronbach’s α>0.7). SAHLPA-23 was more highly correlated with all study variables than SAHLPA-18. Although both instruments displayed acceptable discriminative power, SAHLPA-23 had better accuracy than SAHLPA-18 (DeLong’s method: ΔAUC=0.09, Z=3.36; P<0.001). The SAHLPA-23 is an independent, feasible and innovative tool for estimation of health literacy in the Portuguese adult population.

Type
Development
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s) 2018

The concept of literacy has evolved over recent decades. In the late 20th century literacy was mainly understood in terms of reading and writing skills, but the concept has been extended and now includes the ability to interpret texts, perform calculations and participate actively in society (Organization for Economic Co-Operation and Development and Statistics Canada, 2011; Sorensen et al., Reference Soares2012). As one would expect, there is wide evidence of a positive correlation between education and literacy, although less educated individuals may also be capable of executing complex literacy tasks as a result of regular reading and writing (Benavente et al., Reference Benavente, Rosa, Costa and Ávila1996; Sorensen et al., Reference Soares2012).

As patients should be able to understand and manage relevant health information, literacy is an important issue in healthcare (McCray, Reference McCray2005; Barber et al., Reference Barber, Staples, Osborne, Clerehan, Elder and Buchbinder2009; Chinn, Reference Chinn2011; Mårtensson and Hensing, Reference Mårtensson and Hensing2012; Sykes et al., Reference Sorensen, Van den Broucke, Fullam, Doyle, Pelikan, Slonska and Brand2013). Health literacy may be defined as the ability to obtain, process and understand information about health and health services as necessary to make appropriate health decisions (Sorensen et al., Reference Soares2012: 3).

Studies of health literacy are relevant to health authorities worldwide, since low health literacy is associated with a higher morbidity and mortality and lower treatment adherence; it is also associated with a lower capacity for making health decisions, managing health information and using medication (Dewalt et al., Reference Dewalt, Berkman, Sheridan, Lohr and Pignone2004; Berkman et al., Reference Berkman, Sheridan, Donahue, Halpern and Crotty2011; Zhang et al., Reference Zamora and Clingerman2014). Previous research has shown that low health literacy is more prevalent amongst older people, people with low income and people with few years of education, so these factors contribute to health inequalities (Zamora and Clingerman, Reference Youden2011; Bostock and Steptoe, Reference Bostock and Steptoe2012; Raynor, 2012; Findley, Reference Findley2015; Greenhalgh, Reference Greenhalgh2015). Even in developed countries a high proportion of the population has difficulty managing health information. For instance, the European Health Literacy Project (HLS-EU Project, 2009–2012), which collected data from 8000 participants in eight European countries (Austria, Bulgaria, Germany, Greece, Ireland, the Netherlands, Poland and Spain), revealed limited health literacy skills in 47.5% of the people evaluated (HLS-EU, 2009–2012). The HLS-EU-Questionnaire (HLS-EU-Q) comprises 47 self-report items, to which responses are given using a four-point scale. This questionnaire was used to measure participants’ perception of the difficulty of various health-related tasks. Threshold values were defined, and the scores classified into four health literacy categories: ‘inadequate’, ‘problematic’, ‘sufficient’ and ‘excellent’. Data collected with the HLS-EU-Q in Portugal showed low rates of health literacy: 61.5% of the 1001 participants were categorised as having insufficient health literacy (HLS-EU Project, 2014). Results from another country, mentioned in the report of the National Assessment of Health Literacy of America’s Adults, indicated that 35% of the 18 186 adults assessed had unsatisfactory health literacy skills (National Center for Education Statistics and US Department of Education, 2003). This study categorised participants using a set of performance levels (Below Basic, Basic, Intermediate and Proficient) on the basis of performance on prose, document and quantitative tasks. Prose tasks were related to the knowledge and skills needed to search, comprehend and use information from continuous texts; document tasks tapped the ability to search, comprehend and use information from non-continuous texts in various formats; and quantitative tasks were related to the identification and performance of computations, either alone or sequentially, using numbers embedded in printed material (National Center for Education Statistics and US Department of Education, 2003). The Australian Adult Literacy and Life Skills Survey reported that 59% of the 15 105 participants had inadequate health literacy skills (Australian Bureau of Statistics, 2006). The survey evaluated knowledge and skills in four literacy domains: prose, document, numeracy and problem solving. A fifth indicator, health literacy, was also evaluated (Australian Bureau of Statistics, 2006).

Various tools and methodological approaches have been used to evaluate health literacy (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Collins et al., Reference Collins, Currie, Bakken, Vawdrey and Stone2012; Haun et al., Reference Haun, Valerio, McCormack, Sørensen and Paasche-Orlow2014; Duell et al., Reference Duell, Wright, Renzaho and Bhattacharya2015), such as:

  • The Rapid Estimate of Adult Literacy in Medicine (REALM, 1991): participants are required to read and recognise health words (66 items, 3 min).

  • The Test of Functional Health Literacy in Adults (TOFHLA, 1995): participants are required to perform a modified cloze procedure (health-related information) (67 items, 18–22 min).

  • The Newest Vital Sign (NVS, 2005): participants are required to answer comprehension questions about the nutrition label for an ice cream (six items, around 6 min).

  • The Short Assessment of Health Literacy - Spanish and English versions (SAHL-S; SAHL-E, 2010): participants are required to read and recognise health words (18 items, 2-3 min).

As mentioned before, health literacy tools can be prose, oral or numeracy tasks, and may include self-evaluation items, which may influence their accuracy (Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014; Morrison et al., Reference Morrison, Schapira, Hoffmann and Brousseau2014). Several limitations of the existing health literacy tools have been identified, for example, the length of time and cost of administering them; in addition, these tools are likely to be unsuitable for routine use in healthcare consultations. The patients’ screening of health literacy is preferentially recommended in the first clinical appointment to ensure that health-related information is comprehensible for all subjects through simple, intelligible and effective communication (Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014; Hersh et al., Reference Hersh, Salzman and Snyderman2015). The evaluation of other potentially relevant social aspects, such as participants’ beliefs and behaviours is also considered pertinent when designing health literacy tools (Guzys et al., Reference Guzys, Kenny, Dickson-Swift and Threlkeld2015).

There are only a few published short, self-administered literacy tools, for example, the REALM-Short Form and the Medical Term Recognition Test (METER) (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012; Collins et al., Reference Collins, Currie, Bakken, Vawdrey and Stone2012; Haun et al., Reference Haun, Valerio, McCormack, Sørensen and Paasche-Orlow2014; Paiva et al., 2014; Duell et al., Reference Duell, Wright, Renzaho and Bhattacharya2015; Guzys et al., Reference Guzys, Kenny, Dickson-Swift and Threlkeld2015). Also, there are a few short health literacy tools specifically validated for use with European Portuguese speakers (Benavente et al., Reference Benavente, Rosa, Costa and Ávila1996; Soares, Reference Schmidt and Retelsdorf2010; Paiva et al., 2014). For instance, Paiva et al. (2014) have validated a version of the short health literacy assessment tool METER for use in the Portuguese adult population. This self-administered tool is based on REALM and consists of a list of 40 English medical words and 30 non-words which sound like genuine medical terms. Respondents are required to identify the genuine medical words and scores are calculated as the sum of all the correctly identified words. The health literacy categories for the original METER are as follows: low (0–20), marginal (21–34) and functional (35–40) (Paiva et al., 2014). Two versions of SAHL, a 50-item version (SAHLPA-50) and a shorter 18-item version (SAHLPA-18) have recently been validated for use with Brazilian Portuguese-speaking older adults (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012). The SAHLPA items consist of cards with a medical term printed in bold at the top and two association words at bottom, only one of which is associated with the medical term printed above. First, the respondent is required to read the medical term aloud, then the interviewer reads the two association words and finally the respondent has to choose and read aloud the correct response. Answers are only judged correct if the respondent both chooses the correct word and pronounces it correctly. The SAHLPA-50 takes 3-6 min to complete and the SAHLPA-18 1–2 min (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012). The choice of a health literacy tool for any given situation will depend on several factors, such as the required accuracy and time available for the assessment (Collins et al., Reference Collins, Currie, Bakken, Vawdrey and Stone2012; Haun et al., Reference Haun, Valerio, McCormack, Sørensen and Paasche-Orlow2014; Duell et al., Reference Duell, Wright, Renzaho and Bhattacharya2015). One of the advantages of the SAHL is that it is particularly suitable for evaluating individuals with low health literacy. It has also demonstrated higher reliability than other health literacy indices (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014). The Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012) short version of SAHL is smaller than METER (18 versus 40 items) and this is possible because the SAHL procedure implies respondents’ semantic understanding of medical terms, rather than just requiring them to distinguish medical terms from non-words (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012; Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014; Paiva et al., 2014). The SAHL has been validated in Brazil, but this does not guarantee that it is suitable for assessing the health literacy of the Portuguese population, as there are linguistic differences between Brazilian and European Portuguese as well as social and cultural differences between the two countries (Costa et al., Reference Costa, Duggan and Bates2007; Soares, Reference Schmidt and Retelsdorf2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

As mentioned previously the HLS-EU project (2014) showed that the Portuguese population has a low rate of health literacy and this suggests there is a real need to increase the range of health literacy tools available at national level. Health literacy tools that are simpler and cheaper to administer are more suitable for used in healthcare settings (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012; Paiva et al., 2014). The Portuguese government has recently created a national Programme of Education for Health, Literacy and Self-Care (Law – Despacho no. 3618-A/2016); this demonstrates that the decision-makers have recognised the population’s low level of health literacy and the need to improve it. In addition, Portuguese, taking the European, Brazilian, African and Asian varieties together, is the world’ seventh most commonly spoken language (with around 200 million of speakers) (Nations Online Project, Reference Zhang, Terry and McHorney2017), which provides another reason to develop Portuguese-language health literacy tools.

The evidence and information reviewed above, together with the fact that public health sector resources remain very limited in Portugal (Fernandes and Nunes, Reference Fernandes and Nunes2016) suggest that there is an urgent need for development of short health literacy tools suitable for the Portuguese population. Hence, the objectives of this study were to adapt, improve and validate a short, self-administered version of SAHL for European Portuguese-speaking adults.

Methods

We present two versions of the SAHL, the original SAHLPA-18 adapted from Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012) and a newly developed adaptation, the SAHLPA-23. This section describes the procedure used to adapt the SAHLPA, the method of administration and the scoring of the new tool (SAHLPA-23). The recruitment and inclusion criteria and the protocol for a brief cognitive evaluation are also described. A comprehension questionnaire on a health document was administered and direct and indirect literacy indices were defined and used to assess the concurrent validity of the new tool. Finally, the statistical analysis procedures are described.

Short health literacy tool: SAHLPA-18 and SAHLPA-23

The SAHLPA-23 (Table 1) consists of the recently validated SAHLPA-18 for Brazilian Portuguese-speaking older adults (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012) plus an additional five items dealing with medication-related literacy issues (SAHLPA-23). The inclusion of five new items was expected to improve the psychometric proprieties of the tool, in particular its specificity (Pander Maat et al., Reference Paiva, Silva, Severo, Ferreira, Santos, Lunet and Azevedo2014).

Table 1 Tested tool: Short Assessment of Health Literacy for Portuguese-Speaking Adults (SAHLPA-23)

a The five new items specifically selected from the Quality Review of Documents template (European Commission, 2009). SAHLPA-23 corresponds to SAHLPA-18 (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012) plus the five new items. SAHLPA-23 was the only tool administered.

The words in italics correspond to those presented, written in Portuguese.

The original 18-item tool was validated in a sample of participants with limited education (74.4% of the sample had ⩽seven years of schooling; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012), making it suitable for used in the Portuguese population, which has limited health literacy (Portuguese Institute of Statistic, Reference Pires, Vigário and Cavaco2011; HLS-EU Project, 2014). Each SAHPLA-23 item requires the respondent to select, from two association words, the word which best describes the target medical term (eg, ‘prostate’ may be correctly described as a ‘gland’ or incorrectly described as ‘circulation’). The respondent may also choose a third response option: ‘do not know’ (Table 1). Two pharmacists and one linguist were involved in adapting the SAHLPA-18; in particular they were, required to check possible linguistic differences between European and Brazilian varieties of Portuguese (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012). Few changes were required, perhaps due to the technical nature of the items.

Additional items: SAHPLA-23 and scoring

The introduction of new items during adaptation of health literacy tools has been reported in prior studies (Lee et al., Reference Lee, Bender, Ruiz and Cho2006; Richman et al., Reference Raynor2007). The five new items were selected from the Quality Review of Documents template for European package leaflets (PLs), based on their high frequency in written information designed for patients and consensus amongst the research team. All the new target terms belong to the healthcare domain and are widely used at a national and international level (European Medicine Agency, 2016). They are: ‘active ingredient’ (substância ativa), ‘adverse reactions’ (efeitos secundários), ‘excipient’ (excipiente), ‘fertility’ (fertilidade) and ‘precautions’ (precauções). Two of the five new items were deliberately designed with two-word targets in European Portuguese (‘active ingredient’ and ‘adverse reactions’) as it was thought that longer targets would provide better discrimination between levels of health literacy. The positioning of the five additional items in the tool is random (Table 1). All the correct responses score 1 point and all other responses score 0 points, thus SAHLPA-18 and SAHLPA-23 scores range between 0 and 18 points and 0 and 23 points, respectively. Scores on the items included in the original version of the tool (SAHLPA-18) were compared with the scores of SAHLPA-23 in order to determine which offered the better measure of the construct of health literacy.

Collection of socio-demographic data and comprehension questionnaires

A set of additional self-administered tools was used to collect socio-demographic data (ie, age, gender, income level, education, employment and civil status) and assess comprehension of health-related documents. Comprehension was evaluated with two questionnaires on the comprehension of PLs of medicinal products. These last two questionnaires were developed in accordance with the European guidelines on medicine PLs clarity and comprised eight questions (Appendix 1) (European Commission, 2009; Portuguese Institute of Statistics, Reference Pires, Vigário and Cavaco2011). They were scored as follows: incorrect answer: 0 points; partially correct answer containing some incorrect information: 1 point; partially correct answer without any incorrect information: 2 points; correct answer: 3 points; thus the maximum score was total score for each was 24 points. These instruments, as well as the structured protocol for evaluation of PL comprehension, were developed and validated in a previous study (Pires et al., Reference Pedhazur and Schmelkin2016). Scores on the comprehension questionnaires, as well the direct and indirect literacy indices (described in the next section) were necessary for the validation of the health literacy tool (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

Direct and indirect literacy indexes

The direct literacy index was composed of the following literacy tasks:

  • The spelling of the words carro (car), vaso (vase), bola (ball), lápis (pencil) and relógio (watch) (maximum score: 5 points).

  • The mental calculus task: 100−7(maximum score=1 point), plus two additional calculation tasks (maximum score=2 points). The two additional calculation tasks were as follows: a sum followed by a subtraction to calculate how much money respondents will get back in a purchase and a division to calculate the price of a certain amount of a product. These last two calculation tasks were adapted from the last national Portuguese literacy study (Benavente et al., Reference Benavente, Rosa, Costa and Ávila1996). The maximum score on the direct index was 8 points.

The indirect literacy index was based on participants’ declarations about their reading and writing habits. It included closed questions about how often participants read material in the following categories: (1) books (six per year, five or six per year, three or four per year; one or two per year, rarely and never; maximum score: 5 points); and (2) magazines or journals (daily, at least once a month, at least once a week, rarely and never; maximum score: 4 points). Participants’ writing habits were also assessed, using questions about how often they performed simple writing tasks (eg, writing phone messages). The response options were as follows: daily, at least once a week, at least once a month, rarely and never (maximum score: 4 points). These questions were also adapted from the last national Portuguese literacy study 1995, reported by Benavente et al. (Reference Benavente, Rosa, Costa and Ávila1996). The maximum total score on the indirect literacy index was 13 points.

The cognitive index

Before administering the SAHLPA-23 and the study questionnaires a brief screening test was administered to detect any cognitive problems (eg, dementia) that might affect participants’ judgement and reasoning capacity. The test consisted of the following questions and simple tasks: (1) ‘What month is it?’ (2) ‘Where are we (what city, town, or village)?’ (3) ‘The researcher is about to say three words in loud voice: please write them down (the words were car, vase and ball),’ (4) ‘How much is 100 minus 7?’ (5) The researcher shows the participant a pencil and asks him or her to write down the name of the object, (6) The researcher shows the participant a watch and asks him or her to write the name of the object and (7) ‘Write down the three words that the researcher said in loud voice a few minutes ago, again.’ These questions and tasks were based on the full version of the Mini-Mental State Examination (Folstein et al., Reference Folstein, Folstein and McHugh1975; Grady, Reference Grady2008) and scored as follows: 1 point each for correct responses to questions 1, 2 and 4-6 (0 for incorrect responses) and 1 point for each correct word on questions 3 and 7 (maximum of 3 points per question). As the aim was to check participants’ ability to recognise and write words pronounced in loud voice spelling errors were not taken into account in scoring the cognitive index; however, they were taken into account when calculating the direct literacy index. We included calculation of 100−7 because this task is not supposed to be difficult, especially for the more literate participants. The maximum score on the cognitive index was 11 points. Importantly, our sample had a mean score of 10.8 in the cognitive index, indicating that the participants’ were likely to have sufficient cognitive capacity to complete the other assessments.

Recruitment and selection of the participants

The participants were a convenience sample recruited from the Lisbon and Tagus Valley and Central Portugal regions between August and December 2014. These administrative regions differ with respect to several characteristics: the former is urban, coastal and has a high population, whereas the latter is rural, inland and has a low population (Pires et al., Reference Pedhazur and Schmelkin2016). An email invitation was sent to various city-hall services, military institutions, fire-fighting departments, public cleaning services, parish centres, residential and nursing homes, and education institutions from the two regions. The inclusion criteria were as follows: age of at least 18 years (the age of legal majority in Portugal; Public Ministry, 1966), willingness to participate, having Portuguese as native language and being able to read and write. A sample of 503 participants was recruited, 49.3% from Lisbon. The required sample size was calculated using the formula given by Krejcie and Morgan (Reference Krejcie and Morgan1970) (Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014) and the population data from the last Portuguese census (Krejcie and Morgan, Reference Krejcie and Morgan1970). A detailed explanation of the calculation of the sample size can be found in an earlier study (Pires et al., Reference Pedhazur and Schmelkin2016).

Administration of the short health literacy tool

The tool (SAHLPA-23; see Table 1) was designed to be self-administered, although participants were given oral instructions on how complete it. It was distributed to all participants and completion was overseen by a researcher. Immediately after the questionnaires had been distributed participants were instructed to read the term written in the first column and to mark the option that was most closely related to it, or the ‘do not know’ option with a cross (Table 1). Participants were told that they could ask for clarification if they were unsure about how to complete the questionnaire, but the overwhelming majority (99%) had no problems. Participants were given a maximum of 15 min to complete the SAHLPA-23. This time period was defined based on the results of a pilot test with five elderly participants with limited education. Only 11 participants (5.9%) declared to have spent the 15 min, thus the period allowed was deemed adequate to allow respondents to complete the tool without rushing. The study materials were randomly distributed to the participants to avoid an uneven use of time.

Statistical analysis

Descriptive statistics were calculated. As the percentage of missing values was below 3% no imputation method was applied. A multi-step validation of the SAHLPA-23 was carried out as follows: (1) exploratory factor analysis (EFA) was carried out [Kaiser–Meyer–Olkin Measure (KMO) of sampling and adequacy and the Bartlett’s Test of Sphericity]. Principal component factor analysis with oblique rotation (Direct Oblimin with the delta parameter equal to 0) was conducted based on eigenvalues values (Kaiser’s criterion: eigenvalues ˃1 and the scree plot as the extraction strategy) (Pedhazur and Schmelkin, Reference Pander Maat, Essink-Bot, Leenaars and Fransen1991). (2) The reliability of both the SAHLPA-18 and SAHLPA-23 was expressed as Cronbach’s α0.(3) The discriminatory ability of both tests was examined using receiver operating characteristic (ROC) curves and area under the curve (AUC). The optimal cut-off point was defined as the maximum value of the Youden index, calculated as: J=max [sensitivity c+specificity c–1], where c represents all possible criterion values (Youden, Reference Tavakol and Dennick1950). The difference between AUCs was assessed using DeLong’s non-parametric approach (DeLong et al., Reference DeLong, DeLong and Clarke-Pearson1988).

A cut-off score for adequate health literacy was identified using ROCs for the established literacy tasks. This procedure resulted in exclusion of some items and the definition of a cut-off score for distinguishing between adequate and inadequate health literacy. (4) Spearman’s rank test was used to assess correlations between score on the new short health literacy tool (SAHLPA-23) and the construct variables. Construct validity was evaluated by calculating correlations between SAHLPA-23 scores and the following variables: years of schooling, cognitive index, literacy tasks, reading and writing habits, score on the comprehension questionnaire, and the socio-demographic variables. The Statistical Package for Social Sciences (SPSS) version 22 (IBM SPSS Inc., Chicago, IL, USA) was used for all analyses except for ROC analyses, which were conducted using MedCalc version 14.8 (MedCalc Software, Mariakerke, Belgium). All statistical tests were two-tailed (α level<0.05).

Results

Here, we present the socio-demographic data, data from the direct and indirect literacy indices, cognitive assessment and the scores of SAHLPA-23 and SAHLPA-18. We also compare the psychometric properties of SAHLPA-23 and SAHLPA-18 using EFA, Cronbach’s α, tests of construct validity, Spearman’s rank test, outliers, ROC curves and AUCs.

Socio-demographic data

The final sample consisted of 484 adults as 19 participants from the original sample had to be excluded for statistical reasons. The participants were mainly from city-hall services (26.4%), the military (21.3%) and university undergraduates (non-biomedical studies) (19.9%). The remaining participants were evenly distributed across the other institutions. Overall, 49.3% of the participants were from Lisbon and Tagus Valley, 53.1% (n=256) were male, with a mean age of 38.7 year (SD=18.2, range 18-88) and an average of 10.3 years (SD=4.8) of schooling (Table 2). Table 3 shows the scores on the study variables.

Table 2 Characteristics of the participants (N=484)

n=number of participants.

a The sex of two participants was not registered.

b The average number of years of schooling by each one of the three education groups were as follows: average=4.2, SD=1.3 (⩽6); average=10.8, SD=1.4 (7–12); and average=15.4, SD=1.4 (>12 years of schooling).

Table 3 Mean scores and standard deviations for the study variables

SAHLPA-18 and SAHLPA-23=Short Assessment of Health Literacy for Portuguese-Speaking Adults with 18 and 23 items, respectively.

SAHLPA-23 was the only administered tool, that is, SAHLPA-18 is referent to a separate analyse comprising the results for the items of the original version of Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

EFA of SAHLPA-23 and SAHLPA-18

The SAHLPA-23 data yielded a good KMO score (0.89) and Bartlett’s sphericity test was significant [χ 2(253)=2764.4, P<0.001], indicating that the correlation matrix was suitable for factor analysis. According to the ratio-of-first-to-second-eigenvalues-greater-than-three rule (Slocum-Gorit and Zumbo, 2011) the SAHLPA-23 displayed a one-dimensional structure (5.97/1.78=3.36).

The SAHLPA-18 was subjected to a similar factor analysis. The data displayed a KMO score (0.87) and Bartlett’s sphericity test was significant [χ 2(153)=1963.6, P<0.001], indicating that the correlation matrix was suitable for factor analysis. As with the SAHLPA-23, the ratio-of-first-to-second-eigenvalues-greater-than-three rule also indicated that a one-dimensional structure (4.97/1.52=3.32). Visual inspection of the scree plots for both instruments also revealed a multidimensional factor structure (Appendix 2).

Cronbach’s α: SAHLPA-23 and SAHLPA-18

Internal consistency was estimated via Cronbach’s α (Appendix 3); 0.70 was regarded as the acceptable minimum (Tavakol and Dennick, Reference Sykes, Wills, Rowlands and Popple2011). The corrected item-total correlations and α-if-item-deleted values were also examined for SAHLPA-23. There were only two items where removal increased Cronbach’s α (items 15 and 19, ie, none of the five new items) and the resulting improvements in Cronbach’s α were minimal (<0.01) so all the items were retained. Cronbach’s α was 0.81 for the SAHLPA-18, indicating good reliability (Appendix 3).

Construct validity and Spearman’s rank test

As the data were not normally distributed Spearman’s rank order correlations were used to assess construct validity. The SAHLPA-23 was positively correlated (all P’s<0.05) with schooling [r s(482)=0.537], the cognitive index [r s(482)=0.374], the comprehension questionnaire [r s(482)=0.561], direct measures of literacy [r s(482)=0.308] and indirect measures of literacy [r s(482)=0.234], suggesting that the SAHLPA-23 had good construct validity. The SAHLPA-23 was more highly correlated with all relevant study variables, except the direct and indirect measures of literacy, than the SAHLPA-18 (Table 4).

Table 4 Spearman’s correlation coefficients between the SAHLPA-23/SAHLPA-18 and study variables

SAHLPA-18 and SAHLPA-23=Short Assessment of Health Literacy for Portuguese-Speaking Adults with 18 and 23 items, respectively.

The study variables are the selected variables for the validation procedure. SAHLPA-23 was the only administered tool, that is, SAHLPA-18 is referent to a separate analyse comprising the results for the items of the original version of Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

a Three education groups were considered: ⩽6, 7–12 and >12 years of schooling.

***P<0.001 (n=484).

Spearman’s rank test and Fisher r-to-z transformation

Fisher r-to-z transformation was used to assess the significance of the difference between the correlation coefficients relating the SAHLPA-23 and SAHLPA-18 to variables of interest. The Spearman’s correlation coefficient for association with comprehension questionnaire score was larger in the case of the SAHLPA-23 than the SAHLPA-18 (Z=2.89, P=0.003). Similarly, the association with the direct literacy index was larger in the case of the SAHLPA-23 than the SAHLPA-18 (Z=2.26, P=0.023). The rest of the variables were similarly associated with both instruments (P>0.05).

Outliers, ROC curves and AUCs

In accordance with classical data reduction procedures (Bain and Engelhardt, Reference Bain and Engelhardt1992) all the SAHLPA-23 scores more than 2 SD above or below the mean of each independent groupFootnote 1 were considered outliers. Altogether, 19 outliers (3.78% of the sample) were removed; eight from the adequate health literacy group and 11 from the inadequate health literacy group. Thus, the final sample comprised 484 participants.

Participants scoring <20 (80%) (maximum score=24) on our structured protocol for comprehension of PLs were defined has having inadequate health literacy. This cut-off score was based on the recommendations of the European Medicines Agency as expressed in the Guidelines on the readability of the labelling and package leaflet of medicinal products for human use (European Commission, 2009). Using this cut-off 310 (61.6%) participants were classed as having inadequate health literacy and 193 (38.4%) were classed as having adequate health literacy. As can be seen from Figure 1, the SAHLPA-23 AUC for detection of inadequate health literacy was 0.76 (95% confidence interval [0.73; 0.80], P<0.001), which suggests the tool is a moderately accurate method of detecting inadequate health literacy. The recommended minimum AUC for diagnostic tests is 0.75 (Larner, Reference Larner2015).

Figure 1 Receiver operating characteristic curve for detection of inadequate health literacy (N=484). SAHL=Short Assessment of Health Literacy; SAHLPA=Short Assessment of Health Literacy for Brazilian Portuguese-speaking adults; CI=confidence interval; AUC=area under the curve.

The best SAHLPA-23 cut-off value for detection of inadequate heath literacy was ⩽19 (Youden’s J=0.42; sensitivity=68.2%; specificity=73.5%) (Appendix 4). This cut-off value represents ~83% of the maximum possible score and resulted in 53% participants (257 out of 484) being classified as having inadequate health literacy. The 50-item Brazilian version of SAHLPA has a similar optimal cut-off, 42 (84% of the maximum possible score) (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

Comparison of AUCs: DeLong’s method

Comparison of the AUCs using DeLong’s method revealed that the SAHLPA-23 was a more accurate method of discriminating levels of health literacy than the SAHLPA-18, ΔAUC=0.09, Z=3.36, P<0.001. The SAHLPA-23 also had a larger AUC than the direct measure of literacy, ΔAUC=0.10, Z=3.40, P<0.001 and the indirect measure of literacy, ΔAUC=0.15, Z=4.70, P<0.001 (Figure 2).

Figure 2 Receiver operating characteristic curves for detection of inadequate health literacy between SAHLPA-23 and other variables (N=484). SAHL=Short Assessment of Health Literacy; SAHLPA=Short Assessment of Health Literacy for Brazilian Portuguese-speaking adults.

Years of formal schooling (<7; 7-12; >12) had the highest AUC, which was higher than the AUC for the SAHLPA-23 (Z=2.49, P=0.02). Comparisons of the ROC curves for SAHLPA-23 and other instruments are shown in Table 5.

Table 5 Pairwise comparison of area under the curves (AUCs) between SAHLPA-23 and other variables (N=484)

SAHLPA-23=Short Assessment of Health Literacy for Portuguese-Speaking Adults with 23 items.

*P<0.05; ***P<0.001.

Discussion

The aim of this study was to assess the suitability of the SAHLPA-23 and SAHLPA-18 for use as brief, self-administered tools for evaluating health literacy in the adult Portuguese population. Both SAHLPA-23 and SAHLPA-18 showed suitable psychometric properties: one-dimensional factor structure, Cronbach’s α >0.7, mean-corrected item-total correlation >0.7 and high positive correlations with convergent variables. Moreover, the results are in line with the findings of Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012). Although both tools showed adequate reliability and good construct validity, the SAHLPA-23 is a better method of assessing health literacy as it discriminates more accurately between inadequate and adequate levels of health literacy (ie, has a higher AUC). It was confirmed that the addition of five new items to the SAHLPA-18 was advantageous.

Administration of SAHLPA-23

The SAHLPA-23 appears suitable for self-administration as most participants had no questions about the completion procedure and took less than 15 min to complete the assessment; furthermore no responses had to be excluded due to failure to comply with instructions. The fact that the tool is self-administered may contribute to the length of time it takes to complete, which is somewhat longer than for other tools, such as SAHL-S and -E, which only requires 2–3 min to complete (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Haun et al., Reference Haun, Valerio, McCormack, Sørensen and Paasche-Orlow2014). In comparison with previous versions of the same tool (Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012) the SAHLPA-23 has the advantage that administration does not depend on the direct involvement of a health professional/researcher. Participants were required to mark their chosen response option with a cross rather than reading it aloud (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012) as a result recordings and transcriptions of the oral answers were not required. It is worth noting that recordings of oral answers should be checked by linguistic experts to ensure that the medical terms are pronounced correctly, but we have not found reference to such a procedure in reports on the other versions of this tool (Lee et al., Reference Lee, Stucky, Lee, Rozier and Bender2010; Apolinario et al., Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

Health literacy findings from SAHLPA-23

Around half the participants (53%) were classed as having inadequate health literacy with the SAHLPA-23. This figure is slightly higher than that obtained in the last Portuguese health literacy study, in which only 38.5% of the 1001 participants were classified has having adequate or excellent health literacy (HLS-EU Project, 2014). This discrepancy may be due to the difference between the tools used (a questionnaire versus a short health literacy tool). Importantly, a recent validation of NVS in the Portuguese population yielded almost identical findings to this study, with 54% of the participants classed as probably having inadequate health literacy (Soares, Reference Schmidt and Retelsdorf2010). This suggests that the SAHLPA-23 may be positively correlated with other health literacy tools previously validated for use in the Portuguese population. These findings provide additional evidence that the SAHLPA-23 is suitable for screening health literacy in the Portuguese population.

Direct and indirect indices of literacy

Given that the mean score on the indirect literacy index (closed questions on reading and writing habits) was higher than the mean score on the direct literacy index (numeracy tasks), it is likely some participants overestimated their reading and writing habits (Schmidt and Retelsdorf, 2016). Additionally, writing habits were evaluated based on self-reported frequency of performance of simple tasks (eg, writing phone messages), which may reduced the specificity of this variable as an index of health literacy.

PL comprehension questionnaire

This variable contributed to support construct validity. The mean score for this prose task was low, with only around half of the participants scoring ⩾75% (Pires et al., Reference Pedhazur and Schmelkin2016). This suggests that the documents on which it was based were too complex.

Study strengths

In line with other studies (Collins et al., Reference Collins, Currie, Bakken, Vawdrey and Stone2012; Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014), the following validation requisites were met:

  • a valid construct was defined;

  • Cronbach’s α was appropriate;

  • a large sample was enrolled;

  • ROC curves were calculated and analysis of the AUCs confirmed that the tools showed adequate discrimination

  • a cut-off for discriminating between probable adequate and inadequate health literacy was defined;

  • an appropriate number of questions was defined;

  • the administration procedure has been documented;

  • the method of calculating the score has been described;

  • the study costs were low;

  • the time needed to administer the tool was acceptable.

Study limitations

To validate a new tool it is important to compare the results with those obtained from administration of a ‘gold standard’ assessment, for example, a previous validated tool such as NVS (Collins et al., Reference Collins, Currie, Bakken, Vawdrey and Stone2012; Altin et al., Reference Altin, Finke, Kautz-Freimuth and Stock2014). This was not possible due time constraints and the difficulty of recruiting participants willing to complete both assessments. Additionally, the cognitive assessment needed to be more robust to eliminate the possibility that the sample included cognitively impaired participants; it would have been better to use a validated tool such as the ‘Mini-Mental State Examination’ (Folstein et al., Reference Folstein, Folstein and McHugh1975; Grady, Reference Grady2008).

Previous analyses of health literacy (Guzys et al., Reference Guzys, Kenny, Dickson-Swift and Threlkeld2015) have taken into account socio-demographic data and social context (eg, social habits and health beliefs) and this has been done in previous studies of health literacy and it is recommended practice to contextualise evaluation of health literacy by carrying out a concurrent social evaluation. It is also recommended that the evaluation include a more robust assessment of reading and writing habits (eg, asking participants for the titles of books they have read in the previous year or collecting writing samples).

Schooling was the best predictor of health literacy in our study, that is, the AUC for schooling was significantly higher than the AUC for SAHLPA-23. This result might or might not be replicated in other samples, as our sample was not statistically representative of the Portuguese population. This means that our results may not generalise to national level. Moreover our low- and medium-education groups had had 4 and 11-12 years of schooling, respectively, leaving a gap in the distribution of years of education. This is mainly due to historical changes in the length of compulsory education in Portugal, which has changed from four years (in 1960) to nine years (in 1986) and eventually 12 years (in 2009).Footnote 2 The number of participants in medium-education group (7-12 years of education) with nine years of schooling may have been low because the sample was recruited from public institutions where posts usually require at least 12 years of schooling and often additional education.

Practical implications

SAHLPA-23 may be particularly useful to Portuguese health professionals and researchers as a quick method of obtaining an indication of health literacy in clinical and research settings, in order to anticipate potential problems with health literacy and facilitate more effective communication. Where possible socio-demographic data, such as years of education, should be collect and analysed together with this literacy measure.

Conclusion

The SAHLPA-23 is an independent, effective and innovative self-administered tool for discriminating between adequate and inadequate health literacy. The SAHLPA-23 is a self-administered tool, unlike previous versions of the tool.

Acknowledgements

The authors thank iMed.ULisboa (Research Institute for Medicines and Pharmaceutical Sciences, University of Lisbon), and the Centre of Linguistics of the University of Lisbon (School of Arts and Humanities, University of Lisbon).

Financial Support

This research was partially funded by Foundation for Science and Technology (FCT) Portugal: grants SFRH/BD/76531/2011, PEst-OE/LIN/UI0214/2013, UID/LIN/00214/2013, UID/DTP/04138/2013. FCT is the national funding agency that supports science, technology and innovation, in all scientific domains, under responsibility of the Ministry for Science, Technology and Higher Education (http://www.fct.pt/fct.phtml.en).

Conflicts of Interest

None.

Appendix 1

Table A1 Template of the questionnaire

Appendix 2

Table A2 Scree plot for the SAHLPA-23 and SAHLPA-18

Appendix 3

Item-total statistics for SAHLPA-23 and SAHLPA-18

Table A3 Item-total statistics for SAHLPA-23

Table A4 Item-total statistics for SAHLPA-18

Appendix 4

Figure A1 Interactive dot diagram with sensitivity and specificity plotted against the criterion value (n=484). SAHL=Short Assessment of Health Literacy.

Footnotes

1 Cut-off value (<20) on the comprehension of PLs assessment (maximum score=24) was used to categorise participants into groups on the basis of health literacy: adequate (n=193; 38.4%) and inadequate (n=310; 61.6%).

2 Law – Decreto-Lei no. 42994/1960 and 85/2009.

Quest=questionnaire.

a A name of a disease was randomly selected based on the indications of the medicine described in the evaluated package leaflet (PL). The first name was always excluded because it was considered to be too accessible to the reader.

b A clinical situation related to a contraindication was randomly selected from the list in the PL (if the medicine did not present contraindications, one of the precautions listed in the PL was selected instead). The first contraindication/precaution was excluded because it was considered too accessible to the reader.

c The name of an active component that interacts with the medicine listed in the PL was randomly selected. The first interaction described in the text was excluded, because it was considered too accessible to the reader.

d A drug adverse reaction listed in the PL was randomly selected. The first drug adverse reaction described in the text was excluded, because it was considered too accessible to the reader.

SAHLPA-18 and SAHLPA-23=Short Assessment of Health Literacy for Portuguese-Speaking Adults with 18 and 23 items, respectively.

SAHLPA-23 was the only administered tool, that is, SAHLPA-18 is referent to a separate analyse comprising the results for the items of the original version of Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

SAHLPA-18 and SAHLPA-23=Short Assessment of Health Literacy for Portuguese-Speaking Adults with 18 and 23 items, respectively.

SAHLPA-23 was the only administered tool, that is, SAHLPA-18 is referent to a separate analyse comprising the results for the items of the original version of Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

SAHLPA-18 and SAHLPA-23=Short Assessment of Health Literacy for Portuguese-Speaking Adults with 18 and 23 items, respectively.

SAHLPA-23 was the only administered tool, that is, SAHLPA-18 is referent to a separate analyse comprising the results for the items of the original version of Apolinario et al. (Reference Apolinario, Braga, Magaldi, Busse, Campora, Brucki and Lee2012).

References

Altin, S.V., Finke, I., Kautz-Freimuth, S. and Stock, S. 2014: The evolution of health literacy assessment tools: a systematic review. BMC Public Health 14, 1207.Google Scholar
Apolinario, D., Braga, C., Magaldi, R.M., Busse, A.L., Campora, F., Brucki, S. and Lee, S.Y. 2012: Short assessment of health literacy for Portuguese-speaking adults. Revista de saúde pública 46, 702711.Google Scholar
Australian Bureau of Statistics. 2006: ALLS: Adult Literacy and Life Skills Study. Retrieved 17 December 2017 from http://www.letsread.com.au/getmedia/96e0d766-3821-4878-a3bb-e9b41c01d347/ABS-2006.pdf.aspx Google Scholar
Bain, L.J. and Engelhardt, M. 1992: Introduction to probability and mathematical statistics. Boston, MA: Duxbury Press.Google Scholar
Barber, M.N., Staples, M., Osborne, R.H., Clerehan, R., Elder, C. and Buchbinder, R. 2009: Up to a quarter of the Australian population may have suboptimal health literacy depending upon the measurement tool: results from a population-based study. Health Promotion International 24, 252261.Google Scholar
Benavente, A., Rosa, A., Costa, A.F. and Ávila, P. 1996: A Literacia em Portugal: Resultados de uma Pesquisa Extensiva e Monográfica. (The literacy in Portugal: results of an extensive and monographic search). Lisboa: Fundação Calouste Gulbenkian e Conselho Nacional de Educação.Google Scholar
Berkman, N.D., Sheridan, S.L., Donahue, K.E., Halpern, D.J. and Crotty, K. 2011: Low health literacy and health outcomes: an updated systematic review. Annals of Internal Medicine 155, 97107.Google Scholar
Bostock, S. and Steptoe, A. 2012: Association between low functional health literacy and mortality in older adults: longitudinal cohort study. BMJ 344, e1602.Google Scholar
Chinn, D. 2011: Critical health literacy: a review and critical analysis. Social Science & Medicine 73, 6067.Google Scholar
Collins, S.A., Currie, L.M., Bakken, S., Vawdrey, D.K. and Stone, P.W. 2012: Health literacy screening instruments for eHealth applications: a systematic review. Biomedical Informatics 45, 598607.Google Scholar
Costa, F.A., Duggan, C. and Bates, I. 2007: A systematic approach to cross-cultural adaptation of study tools. Pharm Pract (Granada) 5, 115124.Google Scholar
DeLong, E.R., DeLong, D.M. and Clarke-Pearson, D.L. 1988: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44, 837845.Google Scholar
Dewalt, D.A., Berkman, N.D., Sheridan, S., Lohr, K.N. and Pignone, M.P. 2004: Literacy and health outcomes: a systematic review of the literature. Journal of General Internal Medicine 19, 12281239.Google Scholar
Duell, P., Wright, D., Renzaho, A.M. and Bhattacharya, D. 2015: Optimal health literacy measurement for the clinical setting: a systematic review. Patient Education and Counseling 98, 12951307.Google Scholar
European Commission. 2009: Guideline on the readability of the labelling and package leaflet of medicinal products for human use. Retrieved 17 December 2017 from http://ec.europa.eu/health/files/eudralex/vol-2/c/2009_01_12_readability_guideline_final_en.pdf Google Scholar
European Medicine Agency. 2016: Product-information templates: quality review of documents human product-information annotated template (English) version 9.1. Retrieved 17 December 2017 from http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/document_listing/document_listing_000134.jsp Google Scholar
Fernandes, A.C. and Nunes, A.M. 2016: Hospitals and the public-private combination in the Portuguese health system. Acta Medica Portuguesa 29, 217223.Google Scholar
Findley, A. 2015: Low health literacy and older adults: meanings, problems, and recommendations for social work. Social Work in Health Care 54, 6581.Google Scholar
Folstein, M.F., Folstein, S.E. and McHugh, P.R. 1975: Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 12, 189198.Google Scholar
Grady, C.L. 2008: Cognitive neuroscience of aging. Annals of the New York Academy of Sciences 1124, 127144.Google Scholar
Greenhalgh, T. 2015: Health literacy: towards system level solutions. BMJ 350, h1026.Google Scholar
Guzys, D., Kenny, A., Dickson-Swift, V. and Threlkeld, G. 2015: A critical review of population health literacy assessment. BMC Public Health 15, 215.Google Scholar
Haun, J.N., Valerio, M.A., McCormack, L.A., Sørensen, K. and Paasche-Orlow, M.K. 2014: Health literacy measurement: an inventory and descriptive summary of 51 instruments. Journal of Health Communication 19, 302333.Google Scholar
Hersh, L., Salzman, B. and Snyderman, D. 2015: Health literacy in primary care practice. American Family Physician 92, 118124.Google Scholar
HLS-EU Project. 2009–2012: The European Health Literacy Project 2009-2012: comparative report on health literacy in eight EU member states. Retrieved 17 December 2017 from http://ec.europa.eu/chafea/documents/news/Comparative_report_on_health_literacy_in_eight_EU_member_states.pdf Google Scholar
HLS-EU Project. 2014: Questionário Europeu de Literacia em Saúde aplicado em Portugal. (European questionnaire of literacy in health applied in Portugal). Retrieved 17 December 2017 from http://pelorim.pt/wp-content/uploads/2015/01/resultados-preliminares-HLS-EU-PT.pdf Google Scholar
Krejcie, R.V. and Morgan, D.W. 1970: Determining sample size for research activities. Educational and Psychological Measurement 30, 607610.Google Scholar
Larner, A.J. 2015: Diagnostic Test Accuracy Studies in Dementia. A Pragmatic Approach. London: Springer International Publishing.Google Scholar
Lee, S.D., Bender, D.E., Ruiz, R.E. and Cho, Y.I. 2006: Development of an easy-to-use Spanish health literacy test. Health Services Research 41, 13921412.Google Scholar
Lee, S.D., Stucky, B.D., Lee, J.Y., Rozier, R.G. and Bender, D.E. 2010: Short assessment of health literacy-Spanish and English: a comparable test of health literacy for Spanish and English speakers. Health Services Research 45, 11051120.Google Scholar
Mårtensson, L. and Hensing, G. 2012: Health literacy – a heterogeneous phenomenon: a literature review. Scandinavian Journal of Caring Sciences 26, 151160.Google Scholar
McCray, A.T. 2005: Promoting health literacy. Journal of the American Medical Informatics Association 12, 152163.Google Scholar
Morrison, A.K., Schapira, M.M., Hoffmann, R.G. and Brousseau, D.C. 2014: Measuring health literacy in caregivers of children: a comparison of the Newest Vital Sign and S-TOFHLA. Clinical Pediatrics (Phila) 53, 12641270.Google Scholar
National Center for Education Statistics and US Department of Education. 2003: The health literacy of America’s adults: results from the 2003 National Assessment of Adult Literacy. Retrieved 17 December 2017 from http://nces.ed.gov/pubs2006/2006483.pdf Google Scholar
Nations Online Project. 2017: Most widely spoken Languages in the World. Retrieved 17 December 2017 from http://www.nationsonline.org/oneworld/most_spoken_languages.htm Google Scholar
Organization for Economic Co-Operation and Development and Statistics Canada. 2011: Literacy for life: further results from the Adult Literacy and Life Skills Study. Retrieved 17 December 2017 from http://www.statcan.gc.ca/pub/89-604-x/89-604-x2011001-eng.pdf Google Scholar
Paiva, D., Silva, S., Severo, M., Ferreira, P., Santos, O., Lunet, N. and Azevedo, A. 2014: Cross-cultural adaptation and validation of the health literacy assessment tool METER in the Portuguese adult population. Patient Education and Counseling 97, 269275.Google Scholar
Pander Maat, H., Essink-Bot, M.L., Leenaars, K.E. and Fransen, M.P. 2014: A Short Assessment of Health Literacy (SAHL) in the Netherlands. BMC Public Health 14, 990.Google Scholar
Pedhazur, E. and Schmelkin, L. 1991: Measurement, design, and analysis: an integrated approach. New York, Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Pires, C., Vigário, M. and Cavaco, A. 2016: Factors influencing subjects’ comprehension of a set of medicine package inserts. International Journal of Clinical Pharmacy 38, 888898.Google Scholar
Portuguese Institute of Statistic. 2011: Census report 2011. Retrieved 17 December 2017 file:///C:/Users/Utilizador/Downloads/D3701-Rel_Censos2011_final%20(1).pdf Google Scholar
Public Ministry. 1966: Decree law no. 47344: Civil Code, Article 122 – Age of majority. Retrieved 17 December 2017 http://www.pgdlisboa.pt/leis/lei_mostra_articulado.php?ficha=101&artigo_id=&nid=775&pagina=2&tabela=leis&nversao=&so_miolo= Google Scholar
Raynor, D.K. 2012: Health literacy. BMJ 344, e2188.Google Scholar
Richman, J.A., Lee, J.U., Rozier, R.G., Gong, D.A., Pahel, B.T and Vann, W.F. Jr 2007: Evaluation of a word recognition instrument to test health literacy in dentistry: the REALD-99. Journal of Public Health Dentistry 67, 99104.Google Scholar
Schmidt, F.T.C. and Retelsdorf, J. 2016: A new measure of reading habit: going beyond behavioral frequency. Frontiers in Psychology 7, 1364.Google Scholar
Slocum-Gorit, S.L. and Zumbo, B.D. 2011: Assessing the unidimensionality of psychological scales: using multiple criteria from factor analysis. Social Indicators Research 102, 443461.Google Scholar
Soares, L. 2010: Literacia em Saúde e Alimentação Saudável: Os novos produtos e a escolha dos alimentos. (Literacy in health and healthy eating: the new products and food choice). PhD Thesis. Escola Nacional de Saúde Pública, Universidade Nova de Lisboa. Retrieved 17 December 2017 https://run.unl.pt/bitstream/10362/4264/1/RUN%20-%20Tese%20de%20Doutoramento%20-%20Luis%20Francisco%20Soares%20Luis.pdf Google Scholar
Sorensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z. and Brand, H., (HLS-EU) Consortium Health Literacy Project European. 2012: Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 12, 80.Google Scholar
Sykes, S., Wills, J., Rowlands, G. and Popple, K. 2013: Understanding critical health literacy: a concept analysis. BMC Public Health 13, 150.Google Scholar
Tavakol, M. and Dennick, R. 2011: Making sense of Cronbach’s alpha. International Journal of Medical Education 2, 5355.Google Scholar
Youden, W.J. 1950: Index for rating diagnostic tests. Cancer 3, 3235.Google Scholar
Zamora, H. and Clingerman, E.M. 2011: Health literacy among older adults: a systematic literature review. Journal of Gerontological Nursing 37, 4151.Google Scholar
Zhang, N.J., Terry, A. and McHorney, C.A. 2014: Impact of health literacy on medication adherence: a systematic review and meta-analysis. The Annals of Pharmacotherapy 48, 741751.Google Scholar
Figure 0

Table 1 Tested tool: Short Assessment of Health Literacy for Portuguese-Speaking Adults (SAHLPA-23)

Figure 1

Table 2 Characteristics of the participants (N=484)

Figure 2

Table 3 Mean scores and standard deviations for the study variables

Figure 3

Table 4 Spearman’s correlation coefficients between the SAHLPA-23/SAHLPA-18 and study variables

Figure 4

Figure 1 Receiver operating characteristic curve for detection of inadequate health literacy (N=484). SAHL=Short Assessment of Health Literacy; SAHLPA=Short Assessment of Health Literacy for Brazilian Portuguese-speaking adults; CI=confidence interval; AUC=area under the curve.

Figure 5

Figure 2 Receiver operating characteristic curves for detection of inadequate health literacy between SAHLPA-23 and other variables (N=484). SAHL=Short Assessment of Health Literacy; SAHLPA=Short Assessment of Health Literacy for Brazilian Portuguese-speaking adults.

Figure 6

Table 5 Pairwise comparison of area under the curves (AUCs) between SAHLPA-23 and other variables (N=484)

Figure 7

Table A1 Template of the questionnaire

Figure 8

Table A2 Scree plot for the SAHLPA-23 and SAHLPA-18

Figure 9

Table A3 Item-total statistics for SAHLPA-23

Figure 10

Table A4 Item-total statistics for SAHLPA-18

Figure 11

Figure A1 Interactive dot diagram with sensitivity and specificity plotted against the criterion value (n=484). SAHL=Short Assessment of Health Literacy.