Introduction
Nonadherence to treatment is a serious concern that affects the positive management and prognosis of patients with severe mental illness (SMI). Pharmacotherapy is essential for the successful management of these patients. For a person with a psychiatric illness, keeping to a prescribed regimen of medication is critical. The illness of a patient who fails to adhere will inevitably worsen and persist, which will lead to a deterioration of the patient’s overall health; such nonadherence carries a substantial economic burden, as well [Reference Jüngst, Gräber and Simons1, Reference Maina, Bechon and Rigardetto2]. This is a major problem for patients around the world. The World Health Organization (2020) has reported that the proportion of medication nonadherence is 50% in patients with chronic diseases [3]. Although researchers have reported a nonadherence rate of 20% to 60%, several studies have found that 40% to 60% of psychiatric patients are partially or wholly nonadherent to their prescription medications [Reference Levin, Tatsuoka and Cassidy4–Reference Sajatovic, Ignacio and West6]. Lack of medication adherence is a common and potent (though modifiable) risk factor for poor outcomes [Reference Depp, Lebowitz and Patterson7]. Hispanics are more likely to be nonadherent to psychiatric medication and other treatments [Reference Carliner, Collins and Cabassa8–Reference Lanouette, Folsom and Sciolla10].
Individuals with SMIs are also susceptible to a variety of physical conditions considered to be risk factors for CVD, such as hypertension, metabolic syndrome, hyperlipidemia, type 2 diabetes, and abdominal obesity, among others [Reference Ayerbe, Forgnone and Addo11, Reference Goodrich, Kilbourne and Lai12]. These conditions tend to be poorly treated, also exerting detrimental effects on SMI patients [Reference Damegunta and Gundugurti13]. Not only CVD prevalence rates but also the prevalence rates for its risk factors are about twice as high in SMI patients (for example, in those with affective disorders) as they are in the general population [Reference Birkenaes, Opjordsmoen and Brunborg14, Reference Vancampfort, Firth and Schuch15].
We could not find an instrument in Puerto Rico that assesses nonadherence or poor adherence to medication; while some surveys have been translated in other countries, none have been translated and validated for our population.
For that reason, we decided to develop a valid instrument for measuring treatment adherence/nonadherence in SMI patients with one or more of the following CVD risk factors: hypertension, obesity or abdominal obesity, and/or a generally unhealthy lifestyle (e.g., eats poorly, smokes cigarettes, is physically inactive). We included patients with these comorbidities because the leading cause of premature mortality in this population is related to CVD and CVD risk factors [Reference De Hert, Dekker and Wood16–Reference Saloojee, Burns and Motala19]. Up to now, no gold standard for measuring treatment adherence in Puerto Rican SMI patients with CVD-related comorbidities exists. Furthermore, the questionnaires that are available are Spanish translations of English-language instruments that have never been psychometrically validated for our population. There is a lack of appropriate and validated instruments that can be used to measure adherence–nonadherence behaviors, especially in Puerto Rico. We designed and tested just such an instrument that measures just such behaviors.
Instrument development requires a sample that has those attributes being assessed [Reference DeVellis20]. For that reason, we used a qualitative approach to provide social and cultural context in the construction of our instrument [Reference Ralat, Depp and Bernal21]. We used a psychometric approach to test the instrument for internal consistency and content validity. To create our instrument, we followed the eight-step process devised by DeVellis [Reference De Hert, Dekker and Wood16] for scale development. Our preliminary published data established that nonadherence behaviors are a complex phenomenon with a variety of patterns [Reference Ralat, Depp and Bernal21]. The categories of our study were used to create this scale. There are barriers to adherence to treatment that are related to the medications for a given individual’s mental illness and barriers to adherence to treatment for those diseases and conditions that are themselves risk factors for CVD. Patients named stigma (toward those with a mental illness), patient- and medicine-related issues, poor family support, and factors related to the patient–provider relationship as barriers adhering to the drug regimen prescribed for their psychiatric condition. For the category of barriers to adherence to medications intended to reduce CVD risk, the participants revealed having certain patient-related reasons, mentioned the fact that healthcare personnel does not always provide adequate follow-up care, and named stigma and the lack of support as additional factors.
The aim of this research was to develop and validate a scale to measure adherence and nonadherence behaviors in a culturally sensitive way, taking into consideration the barriers that prevent a patient from taking his or her medication for the treatment of a psychiatric disorder with a comorbid physical condition. This new scale for SMI patients captures the adherence barriers that were determined in our previous study [Reference Ralat, Depp and Bernal21] and can be used by healthcare professionals in targeting interventions that encourage treatment adherence by considering the needs and characteristics of the individual patient.
Methods
Scale Development and Validation
The Institutional Review Board of the University of Puerto Rico, Medical Sciences Campus, approved the study. Informed consent was obtained from all the subjects involved in the study. The STROBE checklist was used as the reporting guideline. (See Supplementary Table 1). This study consisted of two phases. The scale was in Spanish. The first phase consisted of applying the first five of the eight steps of scale development that are recommended by DeVellis [Reference De Hert, Dekker and Wood16]. To that end, we first determined what we wanted to measure, as was previously detailed in our earlier article [Reference Ralat, Depp and Bernal21].
Second, we generated a set of test items forming a pool of 147 items, all related to the concepts of adherence and nonadherence, as established by the literature [Reference Sajatovic, Ignacio and West6, Reference Depp, Lebowitz and Patterson7, Reference Ralat, Depp and Bernal21]. The items were linked to the categories related to the barriers mentioned in the literature and explored by previous research done by the main author.
Third, we determined the format of the measurement (i.e., a Likert scale). The options offered in that scale consist of “totally disagree,” “partially disagree,” “partially agree,” and “totally agree.”
Fourth, we had the items in the initial item pool reviewed (in terms of content) by a group of nine experts. Each expert rated these items according to meaning, content, clarity, and relevance. Three psychiatrists, one cardiologist, two clinical psychologists, and three clinical social workers comprised the experts. In addition to being acknowledged experts in their individual fields, the members of our panel had multiple years working with the issue of adherence (Fig. 1). A standard review guide that included the definitions of adherence and nonadherence was developed. Our experts rated the relevant of each item in terms of what we intended to measure. They examined the content and face validity of the scale and then gave feedback about each item.
Fig. 1. Experts: Years in practice and years working with the issue of adherence.
Fifth, we included in the preliminary scale the items selected by the experts after determining that each one’s calculated content validity ratio (CVR) was acceptable [Reference Lawshe22]. The number of persons making up the panel of experts corresponded to the minimum value of the CVR for each. In this case, a given item needed a ratio of 0.78 to 1 for it to be retained. This procedure was essential to our maximizing the content validity of the scale. A content validity index (0.83) was calculated for the whole test after the items to be included in the first version had been identified; that index indicated that 83% of the items that were included in the instrument were acceptable. Our preliminary scale had 40 items; the CVRs of those items ranged from 0.78 to 1.0, which, according to Lawshe [Reference Lawshe22], is adequate.
The sixth through eighth steps were part of the second phase.
In that second phase, then, we, sixth, administered the scale to a sample. Seventh, we evaluated the items using factor analysis. Eighth, we choose the items of the final scale and obtained a Cronbach’s alpha for that scale.
Participants
The first version of the scale was administered to 160 patients who had been recruited from a clinical psychology practice associated with a private academic institution in San Juan and from the regional branches of an outpatient-serving governmental health agency in several cities in Puerto Rico. A social worker or clinical psychologist at each facility invited a given possible candidate to participate in the study. After the initial approach, the PI was notified that she should contact the candidate. The participants answered a questionnaire that elicited sociodemographic information and questions intended to gather mental and physical health data. Data collection was conducted from February 2017 to December 2019. The participants had bipolar disorder (BD), type I, type II, or unspecified; major depressive disorder (MDD); or schizoaffective disorder (SD). This was a convenience sample. All subjects signed written informed consent.
The inclusion criteria were the following: To participate, the individual 1) had to be taking medication for both physical and mental illnesses; 2) could have one or more risk factors for hypertension, obesity, or abdominal obesity (i.e., a body mass index of 30 kgm2 or more), diabetes, or high cholesterol or triglycerides; 3) could have high LDL levels; and 4) could have a generally unhealthy lifestyle that included having a poor diet, smoking cigarettes, and/or being physically inactive. People with substance abuse problems at the moment of the interview or who were in the midst of a suicidal crisis were excluded from the study. We used the mini-mental state examination, version 2 (MMSE-2) to rule out dementia and severe cognitive deterioration as part of the exclusion criteria.
Reliability, Validity, and Statistical Analysis
Descriptive statistics were used to calculate the demographic characteristics of the sample. A one-way X 2 test was used to analyze adherence/nonadherence through the scale. A two-way X 2 test was used to compare scale scores by sex, BD (type I and type II), MDD, and schizoaffective disorders and to compare the scale scores with the participants’ perceptions of their problems with taking medication. A one-way between-groups analysis of variance (ANOVA) was used to analyze the variable of education as it relates to adherence and nonadherence behaviors. We hypothesized that there will be differences between adherence and nonadherence scale scores and several patients’ variables related to sex, diagnosis, self-perception of adherence behaviors, and cognitive functioning.
The scale was tested for internal consistency and content validity. We computed the Cronbach’s alphas for the scale scores. An exploratory factor analysis was conducted to uncover the latent dimensions among the items. We calculated statistics using IBM SPSS version 21 software. Statistical significance was set at a = 0.05, two-sided.
Results
Baseline Sample Characteristics
Forty-six percent of the participants had schizoaffective disorder; 32%, BD; and 22%, major depressive disorder. The mean age of the participants was 45.5 years (SD = 11.1; range 21–60 years). Sixty-three percent were female. In regard to education, 36% had completed high school. The majority of the participants were single (66.3%). Fifty-nine percent of the sample was from an urban area. (See Table 1 for all the sociodemographic characteristics of the sample.) All the participants (100%) reported that they were currently using psychiatric medication, and 21% of them reported having problems taking their medication and not adhering to their medication during the week that they were recruited for the study. (See Table 2 for the psychiatric diagnosis, medical comorbidities, treatment details, and lifestyle characteristics of the participants by gender.)
Table 1. Sociodemographic characteristics
Table 2. Psychiatric diagnosis, medical comorbidities, treatment details, and lifestyle characteristics, by gender
† Forgot to take my medication in the week prior to taking the questionnaire.
Using X 2 analysis, statistically significant relationships were found in the following variables: obesity and gender (Supplementary Fig. 1), stress and gender (Supplementary Fig. 2), past drug use and gender (Supplementary Fig. 3), and exercise and gender (Supplementary Fig. 4). We found a statistical relationship between high levels of stress and having a depression disorder (Supplementary Fig. 5). In addition, we found a statistical relationship between diagnosis and number of medications (Supplementary Fig. 6).
Factor Analysis
As part of the exploratory factor analysis, we used principal component analysis to extract the maximum variance and put it into the first factor to obtain the minimum number of factors. The latter analysis revealed that the scale had five components with eigenvalues greater than 1, accounting for 57.5% of the total variance explained (Table 3). The first component had an eigenvalue of 7413, followed by 2431 for the second, 1506 for the third, 1234 for the fourth, 1205 for the fifth, accounting for 30.89%, 10.13%, 6.28%, 5.14%, and 5.02%, respectively, of the variance. Supplementary Fig. 7 shows the scree plot. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.85 (good). The Bartlett’s test of sphericity was p < 0.001. We included the distribution of answers for each question of the scale (Table 4). No Bias was found in this sample in terms of their responses to the RAS-24.
Table 3. Total variance explained
Extraction Method: Principal Component Analysis.
Table 4. Distribution of answers to the 24 items that remain in the RAS-24
Reliability
The original scale of 40 items had high internal consistency (Cronbach’s alpha = 0.812), making it a very good index (according to Kline, 2005) [Reference Kline23]. However, after the factor analysis, the scale was reduced to 25 items to compact it and make it more flexible for future use. We choose the items that correlated between 0.509 and 0.718, but not values that were too high because extreme multicollinearity and singularity would cause difficulties in determining the unique contribution of the variables to a factor. The Cronbach’s alphas of the deleted items fluctuated from 0.888 to 0.900. When removing one of the items, the Cronbach’s alpha of the final version of the scale was 0.900. Then, the final scale consisted of 24 items. The principal component analysis for the 24 items that remained revealed five components related to adherence and nonadherence behaviors.
Adherence/Nonadherence Behaviors
We analyzed the data using a one-way X 2 test. Different from other studies, in which frequencies of from 20% to a 60% were observed [Reference Levin, Tatsuoka and Cassidy4–Reference Sajatovic, Ignacio and West6], the frequencies observed in this sample showed that 10% of our sample population had positive adherence behaviors and 90% had nonadherence behaviors, X 2 (1, N = 160) = 102.4, p < 0.001. Ten percent of men and 10% of women adhered to their medication regimens. Persons with BD had 12% adherence and 88% nonadherence behaviors. For MDD, 6% had adherence, and 94% had nonadherence behaviors. Those with SD had 11% adherence and 89% nonadherence behaviors.
There was no statistically significant difference between males and females in the scale scores. There were no differences in the scale scores by diagnosis. However, we found significant differences between the scale scores and several patient variables as follows.
We used a one-way, between-groups ANOVA to analyze the variable of education in terms of adherence and nonadherence behaviors. Least significant difference post hoc comparisons examined differences between groups (p < 0.05). Participants were classified into four groups: individuals having 0 to 4 years of education, those having 5 to 8 years of education, those having 9 to 12 years of education, and those having an undergraduate or higher level of education. Data are presented as the mean ± standard deviation Participants with an undergraduate or higher degree were found to be more adherent to medication (1.82 ± 0.384 [95% CI, 1.75–1.89]; p = 0.016) in comparison to the group of participants with 5–8 years of education (2.00 ± 0.000 [95% CI, 1.84–2.16], p = 0.044) and that of the participants with 9–12 years of education (1.95 ± 0.229 [95% CI, 1.87–2.01]; p = 0.016). The differences between these groups were statistically significant, F(3, 156) = 2.80; p = 0.042; η 2 = 0.051. Having had more years of education contributes to positive adherence behaviors. Supplementary Fig. 8 shows the distribution of the different levels of education with their corresponding adherence/nonadherence behaviors
Using a question that explored the difficulties of taking medication, we examined how the scores of our adherence scale matched up (or did not match up) with the perceptions of the participants regarding their own adherence or lack thereof. Before working with the adherence scale, each participant was asked whether he or she had any difficulties taking his or her medication. This question was part of the sociodemographic questionnaire. Our intention in using this question was to assess each participant’s scores regarding adherence in light of his or her own perceptions of the difficulties in adhering to a medication regimen. We analyzed our data using a 2 (problem taking medication) × 2 (measures of adherence/nonadherence behaviors ascertained by the scale) X 2 test. Though only 21% of the participants reported having difficulties taking their medication, the scale scores indicated that 90% of them had nonadherence behaviors (X 2 (1, N = 160) = 4.80; p < 0.05; ϕ = 0.173).
Finally, there was a statistically significant relationship between participants with cognitive impairment and nonadherence behaviors. We used a 2 (adherence/nonadherence behaviors) × 2 (normal/impairment cognitive functioning) X 2 test. Fifty-one percent of nonadherent patient had cognitive impairment (X 2 (1, N = 160) = 3.83; p < 0.05; ϕ = −0.155). A more detailed analysis of cognitive impairment by diagnosis with this sample was published by the main author [Reference Ralat24].
Discussion and Conclusions
We developed and validated a culturally sensitive adherence-to-treatment scale. Compared with other studies [Reference Levin, Tatsuoka and Cassidy4–Reference Sajatovic, Ignacio and West6], the number of nonadherence behaviors in this study differed from what has been found. In our study, there were more participants with nonadherence behaviors. Education was one of the variables with a significant effect on adherence and nonadherence behaviors different from other studies. Adherence rates were higher in those participants with higher levels of education. Cognitive impairment is another variable that could have an influence related to nonadherence behaviors. Using the MMSE-2, we detected that 51% of patients with nonadherence behaviors had cognitive impairment.
This study supports the notion that adherence to medication regimens can be estimated in SMI patients based on the education level of the individual as well as other variables. Different from other studies, in which participants with high levels of education were found to be more adherent (though not statistically significant so), our study found having a high level of education to be a statistically significant variable that was associated with both adherence and nonadherence behaviors [Reference Rolnick, Pawloski and Hedblom25, Reference Gavrilova, Bandere and Rutkovska26].
Prior to their being interviewed, we asked the participants what—if any—difficulties they had in managing their own medication regimens. The goal was to later assess the differences between perceived and actual (as determined by our scale) adherence; the answers to this question revealed that nonadherence behaviors were more frequently practiced than the participants thought them to be. In the literature, several studies have indicated that self-reports (compared to other assessment methods) tend to overestimate adherence behavior [Reference Stirratt, Dunbar-Jacob and Crane27, Reference Atkinson, Rodríguez and Gordon28]. Social desirability is one of the reasons that patients tend to overestimate their effectiveness in managing their own medication regimens. To help remedy this issue, social desirability must be addressed and a validated measure of adherence (one that focuses on the population of interest) used.
The psychometric properties of this particular scale were considered to be very good. To the best of our knowledge, this is the first scale in Puerto Rico to measure adherence and nonadherence to medication regimens in a population of patients with an SMI. Effectively identifying nonadherence behaviors is the first step in developing and, subsequently, promoting, psychosocial interventions that can enhance treatment adherence; such interventions would be an adjunct to pharmacotherapy [Reference Berk, Hallam and Colom29]. Pharmacotherapy is the recommended first-line treatment for SMI patients, but medication adherence is frequently poor, causing relapses and worsening the psychiatric symptoms and general health of these patients [Reference Maina, Bechon and Rigardetto2, Reference Chatterton, Stockings and Berk30]. Our scale identified adherence and nonadherence behaviors in a sample of SMI patients (Table 5). This scale will be known as the Ralat Adherence Scale (RAS-24) and will consist of 24 items aimed at assessing adherence and nonadherence in SMI patients. The RAS-24 will make it possible for healthcare professionals to explore adherence barriers in their patients.
Table 5. The 24-item Ralat adherence scale (RAS-24: English translation)
Persons wishing to use the scale are requested to formally contact the main author. The scale is copyrighted.
Five principal components related to adherence and nonadherence behaviors were identified. The first component included four items related to adherence behaviors. We also identified the reasons for nonadherence, classifying these reasons as being patient-related, medication-related, stigma related, or related to a lack of support from family members. These five components will help healthcare professionals to identify not only the nonadherence behaviors but also the reasons for their existence, thereby enabling these professionals to offer interventions that promote adherence behaviors. When it comes to medication adherence and nonadherence, tailored psychoeducation has been proven to be more effective than generalized education. The scale described in this article offers information about which barriers a given patient has and helps the health professional to create a tailored psychosocial intervention to better manage that patient’s issues with medication; the purpose is to increase the efficaciousness of this kind of intervention by taking into account the individual’s personal characteristic, needs, beliefs, and attitudes [Reference Newman-Casey, Robin and Blachley31].
This study has several limitations. First, the participants were not randomly recruited and are not representative of all the Puerto Rican SMI patients. Second, patients with other chronic mental illnesses (e.g., schizophrenia) were not included in the sample. Third, the reliability of the scale over time (using a test–retest strategy) was not evaluated. More studies with greater numbers of patients and in other populations with the same and other chronic illnesses are needed to test the RAS-24. In the future, measuring test–retest reliability and performing subsequent validation with confirmatory factor analysis are recommended.
Despite the limitations mentioned, the results provide relevant information about the psychometric properties of the RAS-24.
In conclusion, the RAS-24 is a self-administered instrument with very good psychometric properties; it has already yielded important information about nonadherence–adherence behaviors. The scale can help health professionals and researchers to assess patient adherence or nonadherence to a medication regimen. Identifying nonadherence behaviors and their causes in patients with an SMI will aid in the provision of psychoeducational and psychosocial interventions to both these patients and—when applicable—their caretakers, thereby promoting improvements in their (the patients) psychiatric and comorbid conditions.
The scale can be used for future research examining both preventive measures and potential treatments.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2023.527.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The main author received support from Phase II of the Postdoctoral Master of Science in Clinical and Translational Research Program, the Hispanic Alliance for Clinical and Translational Research and the RCMI-Center for Collaborative Research in Health Disparities, from the Medical Sciences Campus, University of Puerto Rico. The authors would like to express their gratitude to the individuals that participated in this study.
The research reported in this publication was supported by the National Institute on Minority Health and Health Disparities (award numbers R25MD007607 and U54MD007600) and the National Institute of General Medical Sciences (NIGMS) of the National Institutes of Health (award number U54GM133807). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures
The author declares no conflict of interest.
Transparency Statement
We confirm that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Open access
