Introduction
Community-Engaged Research (CEnR) and Community-Based Participatory Research (CBPR) have gained popularity over the past decades as valued research approaches [Reference Wallerstein, Oetzel and Sanchez-Youngman1]. Funders and accrediting bodies increasingly require evidence of partnerships between communities and institutions of higher learning and research [Reference Levinton, Green, Wallerstein, Duran, Oetzel and Minkler2]. Based on empirical frameworks, tools, and resources to guide effective partnering, evaluation, and reflexivity have emerged as best practices to successfully support partnership capacity building, meaningful community engagement, and adherence to CEnR and CBPR principles [Reference Ortiz, Nash and Shea3]. Thus, having validated instruments and metrics for evaluating research partnerships and outcomes is crucial. Moreover, such tools should be adapted and translated for pragmatic use with communities from diverse backgrounds and non-English speaking communities.
The National Academy of Medicine (NAM) Organizing Committee on Assessing Meaningful Community Engagement in Health & Health Care Policies & Programs recently released 28 assessment instruments or measures, each providing standard questions or question sets to assess community engagement consistently. These measures are meant to help potential users support the rigorous assessment and evaluation of CEnR and CBPR and map onto the new NAM Assessing Community Engagement Conceptual Model (ACE-CoM) [4]. However, only four were validated with national samples, and even fewer comprehensively covered the domains of the Organizing Committee’s new model [Reference Aguilar-Gaxiola, Ahmed and Anise5]. Moreover, few were available in Spanish, which can be critical for partnerships working with the Latino/a/x community, the largest and youngest racial/ethnic minority group in the US [Reference Krogstad, Passel, Moslimani and Noe-Bustamante6,7]. Among the three identified Spanish measures in the NAM review, two came from a long-term National Institute of Health-funded research initiative, Engage for Equity [Reference Wallerstein, Oetzel and Sanchez-Youngman1]. One is the Encuesta Comunitaria (or Community Engagement Survey in English), a vital focus of this manuscript. The second is its partner survey, Encuesta de Informante Clave (or Key Informant Survey in English), which is directed at principal investigators or project directors to assess the partnership, such as funding source and amount, Institutional Review Board (IRB) training, etc. [Reference Dickson, Magarati and Boursaw8].
One of the four measures identified with national validation and covering most constructs in the NAM ACE-CoM also comes from Engage for Equity, a collaboration between the University of New Mexico Center for Participatory Research (CPR) and national partners [Reference Wallerstein, Oetzel and Sanchez-Youngman1]. In two successive data collection waves of the internet-based survey tool, Engage for Equity Community Engagement Survey (E2 CES) [Reference Oetzel, Zhou and Duran9,Reference Boursaw, Oetzel and Dickson10]. the Engage for Equity team assessed close to 400 partnering teams to identify best-partnering practices associated with outcomes [Reference Oetzel, Boursaw and Magarati11,Reference Duran, Oetzel and Magarati12]. In the second data collection wave, a Spanish Version, Encuesta Comunitaria, was also developed and tested for validation. Two shorter versions of the CES and Encuesta Comunitaria have also been developed, Partnering for Health Improvement and Research Equity (PHIRE) (Oetzel et al., unpublished data [PHIRE available from nwallerstein@salud.unm.edu]) in English and Fortaleciendo y Uniendo EsfueRzos Transdisciplinarios para Equidad de Salud (FUERTES) in Spanish, respectively.
This manuscript provides an overview of the translation and adaptation process of the long and short Spanish versions of Engage for Equity’s Encuesta Comunitaria and FUERTES, respectively. We also provide measurement properties for the Encuesta Comunitaria. Both tools can assess CEnR partnering processes, practices, and outcomes within Spanish-speaking communities in the US and Latin America.
Materials and methods
This study received approval from two IRBs at the University of New Mexico. The first IRB (Protocol #16–098) covered the refinement, Spanish translation, and testing of the E2 CES in Puerto Rico and Nicaragua. The second IRB (Protocol # 19–376) covered the quantitative and qualitative process to shorten and translate the E2 CES.
Overview of original English measure: Engage for Equity Community Engagement Survey (E2 CES)
E2 CES is a valid and reliable tool appropriate for measuring constructs to evaluate CBPR and CEnR partnerships and to apply results for strengthening project processes and outcomes [Reference Wallerstein, Oetzel and Sanchez-Youngman1,Reference Parker, Wallerstein and Duran13]. The most recent version of the E2 CES measurement instrument includes 81 scale items organized across 23 distinct areas within the four domains of the empirical CBPR model, including context, partnership processes, intervention and research actions, and outcomes [Reference Belone, Lucero and Duran14].
The E2 CES was tested in two iterations, in 2015 with 200 partnerships [Reference Oetzel, Zhou and Duran9] and in 2020 with 179 partnerships [Reference Boursaw, Oetzel and Dickson10]. Scales and constructs in the E2 CES have demonstrated strong factorial validity, convergent validity, and internal consistency [Reference Oetzel, Zhou and Duran9,Reference Boursaw, Oetzel and Dickson10]. These findings suggest that the measures of the E2 CES can be used by CEnR/CBPR practitioners and researchers to evaluate CEnR/CBPR partnerships and to advance the science of CEnR/CBPR. See the UNM CPR website E2 tools and resources for the specific items and response options in the Spanish version of the Community Engagement Survey (CPR, https://hsc.unm.edu/population-health/research-centers/center-participatory-research/cbpr-community-engagement/, 2023) and the Engage for Equity website (https://engageforequity.org).
Linguistic and Pragmatic Adaptation of Engage for Equity’s Community Engagement Survey
As CEnR increases in popularity in many partnerships involving non-English speaking community partners within the US and globally, there is a need for tools that are available in other languages. CBPR has strong roots in Latin America, grounded in Freirean dialogical education and dating back to the first participatory action research conference held in Cartagena, Colombia in 1977 [Reference Wallerstein, Duran, Wallerstein, Duran, Oetzel and Minkler15,Reference Collins, Clifasefi and Stanton16], and where many Spanish-speaking countries have long histories of community participation, including on projects with U.S. institutions [Reference Martin, Andrade, Vila, Acosta-Pérez and Canino17]. Spanish-speaking partnerships have been of particular interest, with data from the NIH RePORTER showing that from 2020 through 2024, the number of CEnR or CBPR federal grants involving Spanish-speaking participants increased from 23 on or before December 2020 to 274 in May 2024. However, to our knowledge, no study has developed validated measures for Spanish-speaking partnerships.
While the E2 CES is an advance in partnership measures, this long form, has a high response burden due to the large number of items, making it difficult for partnerships to incorporate into their regular evaluations of their partnering processes and adherence to best practices for CEnR. According to Glasgow and Riley (2013), pragmatic measures should, at minimum, be necessary to stakeholders, have a low burden, be actionable, and be sensitive to change [Reference Glasgow and Riley18]. To develop shorter pragmatic measures of CBPR processes and outcomes, we employed a method to preserve content validity, psychometric properties, and importance to stakeholders of items, scales, and constructs from the original E2 CES [Reference Glasgow and Riley18]. This process is described in detail below (see Figure 1).
Overview of measures and process for translation, adaptation, and refinement.
Overall Spanish translation and cultural adaptation processes
Although Spanish-speaking Latino/a/x populations are considered one ethnic group in the US, they represent a heterogeneous mix of cultures from more than 20 countries in Latin America and the Caribbean [Reference Krogstad, Passel, Moslimani and Noe-Bustamante6,Reference González Burchard, Borrell and Choudhry19]. The E2 CES was translated and adapted from English to a Spanish version, called Encuesta Comunitaria, that could be understood by diverse Spanish speakers in the US (e.g., from Mexico, Cuba, Puerto Rico, etc.) and that would serve as a basis for adaptations to specific cultural contexts. This full-length Spanish version mirrored the multiple scales found in the English E2 CES, which included validated measures across the four domains of the CBPR model, [Reference Oetzel, Sussman, Magarati, Khodyakov, Wallerstein, Wallerstein, Duran, Oetzel and Minkler20] the development of new scales [Reference Boursaw, Oetzel and Dickson10] informed by prior studies, [Reference Oetzel, Zhou and Duran9] and consultation with a national advisory team of community and academic researchers [Reference Wallerstein, Oetzel and Sanchez-Youngman1,Reference Parker, Wallerstein and Duran13].
The translation and adaptation processes were carried out in collaboration with academic experts in social, clinical, and community research based in the mainland US and Puerto Rico. To promote maximum language and cultural equivalency of the measures for the study, independent bilingual CBPR researchers undertook the following recommended published steps [Reference Toma, Guetterman, Yaqub, Talaat and Fetters21–Reference Vujcich, Roberts and Gu23]: (1) the original version was translated into Spanish by a professional, certified translator independent from the original measure development and study team; (2) the Puerto Rican team made some revisions comparing this Spanish translation item-by-item to the original E2 CES English version in ∼12 hours of meetings, including discussing discrepancies and suggestions; (3) the items were back-translated to compare the wording with the original version and to re-formulate those items which did not keep their original meaning; and (4) cognitive debriefing interviews were conducted by Puerto Rican partners with five community participants to identify difficulties in comprehension, validating the translation and adaptation process.
This refined Spanish version was also reviewed by bilingual Engage for Equity team members (PRE, MA, and LM) who had a deep familiarity with the constructs assessed in the English version. These three bilingual researchers, via roundtable discussions, made additional wording changes, and any discrepancies, comments, or concerns were further discussed with the original English E2 CES project Principal Investigator (NW) and the biostatistician on the team. As a result, Encuesta Comunitaria is a culturally and linguistically adapted measure incorporating feedback from individuals representing different U.S. and Latin American communities (i.e., North, Central, and South America, and the Caribbean), as well as the largest Spanish-speaking populations in the U.S., including but not limited to Mexico, Puerto Rico, Cuba, and Colombia.
Testing La Encuesta Comunitaria across sites
In Nicaragua, a nonprofit primary care rural system, AMOS Health and Hope [Reference Chanchien Parajón, Hinshaw, Sanchez, Minkler and Wallerstein24], decided to use and test the Encuesta Comunitaria with their rural community health workers and national training staff to assess and strengthen their partnering practices with each of their village health committees. In one of their semiannual meetings in Managua, the Encuesta Comunitaria was administered in a group (n = 21) format, with community health workers and staff being able to ask questions as they took the survey. As an additional benefit, the medical director developed a qualitative checklist based on the survey questions to encourage group dialog and co-learning about the value of implementing partnering best practices.
The Encuesta Comunitaria measure was also tested in collaboration with community and academic partners in Puerto Rico. Potential participants who were 21 or older and with no cognitive impairment were recruited through convenience sampling. Invitations to participate in the survey were sent to researchers, leaders, and community-based organizations throughout Puerto Rico via emails or physical flyers. Announcements were also available on the Hispanic Alliance for Clinical and Translational Research (Alliance) websites and partnership universities or organizations. The surveys were hand-delivered or mailed to each interested partner organization. In coordination with the Alliance, the Hispanic in Research Capability Endowment delivered a CBPR workshop open to all Puerto Rican community and academic researchers. The thirty-six people enrolled in the workshop completed the Encuesta Comunitaria measure.
Encuesta Comunitaria data analyses
Descriptive statistics were performed for all measures with scale and subscale scores across items and sociodemographic variables of respondents. Consistent with the most recent English version of the CES, data analyses were performed for the same items representing the 23 subscales [Reference Boursaw, Oetzel and Dickson10]. Internal consistency values were assessed for each scale using Cronbach’s alpha. We performed correlations to examine the relationship between items representing the four CBPR model domains, contexts, partnership processes, intervention and research, and outcomes [Reference Belone, Lucero and Duran14,Reference Wallerstein, Duran, Oetzel and Minkler25]. All analyses were performed using SPSS Version 29.
Results
Fifty-seven respondents from Puerto Rico (N = 36) and Nicaragua (N = 21) participated in the study and completed the Encuesta Comunitaria. As seen in Table 1, most participants identified as Latinx (97%), female gender (74%), and were academic partners (61%). Under the CBPR conceptual model domains, Cronbach’s alpha values ranged from 0.72 to 0.88 for Context, 0.74–0.93 for Partnership Processes, 0.90–0.92 for Intervention and Research, and 0.45–0.94 for Outcomes. See Table 2 for a complete list of items from scales, subscales, and Cronbach’s alpha across the four domains in the CBPR conceptual model.
Demographic characteristics of partners completing the Encuesta Comunitaria
For race/ethnicity, respondents had the option to select all that applied. Therefore, total values may not add up to 100%. No respondents identified as Asian, Native Hawaiian, or Pacific Islander.
Descriptive statistics for items, subscales, and scales of practices and outcomes in the Encuesta Comunitaria
Correlations between all subscales are reported in Table 3 across the four CBPR domains. Scales 1–3 represent the CBPR domain Context, and scales 4–11 represent Partnership Processes. Scales 12–15 correspond to Intervention and Research, and scales 16–23 represent Outcomes. Significant correlations were found between and within domains of measures. For scales representing the CBPR domains Context and Partnership Processes, the strongest positive correlations were between partnership capacity and leadership, r = 0.641, p < 0.01, and bridging differences and leadership, r = 0.609, p < 0.01. The strongest correlation between Context and Intervention and Research Action domains was between partnership capacity and synergy, r = 0.478, p < 0.01, and bridging differences and analyses and dissemination, r = 0.433, p < 0.01. For Context and Outcomes, the highest correlation was between partnership capacity and community power in research, r = 0.617¸ p < 0.01, and bridging differences and community integration r = 0.544, p < 0.01.
Correlation matrix for Encuesta Comunitaria (Spanish Community Engagement Survey)
* < 0.05, ** < 0.01. The bolded constructs on the left are the scales.
The strongest relationship between Partnership Processes and Intervention and Research Action was between partnering principles and synergy, r = 0.705 p < 0.01, and community fit and synergy, r = 0.609, p < 0.01. For Partnership Processes and Outcomes, the highest correlation was between leadership and community power in research, r = 0.628, p < 0.01, and community fit and community integration, r = 0.627, p < 0.01. The strongest relationship between the scales in the Intervention and Research and Outcomes domains was between synergy and community integration, r = 0.567, p < 0.01, and synergy and social transformation, r = 0.541, p < 0.01.
Development of the shorter measure: Fortaleciendo y Uniendo EsfueRzos Transdisciplinarios para Equidad de Salud (FUERTES)
Given the length of the original measure and participant burden, additional expert Delphi consultation and psychometric work was performed to shorten the E2 CES from 93 items to 34 items to provide actionable data to partnerships more flexibly and practically. The goal was that an easy-to-use measure would allow partnerships to engage in annual evaluations and collective reflection about what is working well and what to strengthen in the coming year. Furthermore, a shorter measure enhances the uptake of the Spanish and English versions of the E2 CES in the dissemination and implementation phases of the broader E2 “Collective Reflection and Measurement Toolkit” (see tools on engageforequity.org).
Details on shortening the measure can be found elsewhere. Briefly, E2 Think Tank members and E2 community partners participated in a systematic Delphi process of categorizing E2 CES scales and individual items according to their importance for being included in the shorter measure (from “least important” to “most important: needs to be included”). Classical test theory and item response theory were then utilized to select items that allowed for the preservation of psychometric properties [Reference Jabrayilov, Emons and Sijtsma26]. Each E2 CES item was evaluated and ranked in four areas: consistency within a scale or construct, convergent validity, information, and responsiveness to change, resulting in scales often reduced from 3 to 5 items to a single item. Additional input was received from E2 Think Tank members, E2 community partners, and respondents of the original E2 CES measure before finalizing the shorter version, the PHIRE survey.
Spanish translation and adaptation
To develop the short Spanish version, FUERTES, we performed a similar translation and adaptation process used during the development of the Encuesta Comunitaria (see Figure 1). Bilingual members of our team with origins from different parts of Latin America (e.g., Mexico, Cuba, Colombia, and Puerto Rico) with personal and professional experience working with diverse Spanish speakers across various research projects and settings in the U.S. (e.g., clinical, educational, and community) were involved. First, the original PHIRE survey was translated from English to Spanish by two independent CBPR bilingual researchers (JP and PRE). This translation was independently reviewed and further refined by a professional certified translator (DR). After the initial Spanish translation, these two researchers met regularly with additional bilingual CBPR researchers, including members of the national E2 Think Tank. They discussed each item while comparing it to the English version. During six months, members of the bilingual research team (n = 8) met every other week or monthly for 1.5 hours to compare, discuss discrepancies, and arrive at a consensus of items and response options.
To avoid confusion when responding to negative items in scales, we decided to reverse two negative items to the positive sense [Reference Solís Salazar27]. For example, under the Colaboraciones (Partnering) domain, the negative item, “Cuando tenemos conversaciones, con frecuencia nos malinterpretamos” was changed to “Nosotros nos escuchamos.” This resulted in similar changes in the English PHIRE version. For instance, the negative item “When we have conversations, we often misunderstand each other” was changed to “When we have conversations, we mostly understand each other.” These discussions also helped us find more understandable and equivalent response options. Under the Características (Characteristics) domain, we initially started with the response options: de ningún modo, a un modo pequeño, a un modo moderado, a un gran modo, and a un muy gran modo. Through several dialogs in our meetings, we recognized that it was difficult to distinguish these response options and decided to change them to avoid confusion and ensure better delineation among the options. As seen in Table 4, the final response options were nunca, casi nunca, en ocasiones, con frecuencia, and casi siempre. See Table 4 for the full items and response options.
English and Spanish items in the FUERTES measures
Discussion
The present study contributes to the development of validated Spanish and pragmatic measures for CEnR scholarship that are culturally adapted for use in the US and Latin America. The two instruments, Encuesta Comunitaria and its shorter version, FUERTES, can meet the needs for evaluating or assessing academic-community partnering practices and outcomes involving Spanish-speaking partners and communities. Reviewing response means of aggregated data for each scale enables partners to reflect together on which practices their partnership is implementing well; how these compare with national best practices; and ultimately, what practices they could strengthen in subsequent years. Stratification of data by stakeholder groups may enable a deeper dialog into areas of alignment or misalignment. The goal of these instruments is for dialog and action, with the opportunity for annual implementation and reflection/action, not simply to capture one point in time.
Reflexive practices [Reference Wallerstein and Auerbach28] – or the continuous dialog, reflection, and action around research and partnership practices – are among the recognized empirically-based best practices of CEnR and CBPR [Reference Wallerstein, Oetzel and Sanchez-Youngman1,Reference Muhammad, Wallerstein, Sussman, Avila, Belone and Duran29]. Collective reflexive practices have been shown to contribute to capacity building, enhanced adherence to CBPR principles, equitable power-sharing, and partnership sustainability [Reference Wallerstein, Oetzel and Sanchez-Youngman1]. This reflection, facilitated by the use of these measurement instruments can support partnerships in deeper engagement, reflecting on their partnership goals, and their progress towards achieving intended outcomes, including those of health equity and social justice. Using the Encuesta Comunitaria instrument can support partners to assess current performance and progress over time with constructs that correspond to the CBPR and NAM models [Reference Aguilar-Gaxiola, Ahmed and Anise5,Reference Belone, Lucero and Duran14]. The shorter pragmatic instrument, FUERTES, may more easily facilitate annual collective reflection assessments of partnership practices while limiting burden to partners during completion.
Future directions
Future studies using the Encuesta Comunitaria could assess which subdomains best contribute to specific outcomes prioritized by the partnerships, including policy changes, improved health, access to new or enhanced resources, or sustainability of such changes and the partnership itself. Moreover, future studies are also needed to further validate and refine both instruments with consideration of the heterogeneity of the Latino/a/x population. For example, in New Mexico, a state-wide CBPR training, involving the Department of Health, the University of New Mexico, New Mexico State University, and local communities, has produced a module that incorporates PHIRE and FUERTES instruments for diverse collaborative projects to engage in collective evaluation. Discussion is also underway for application with the new NIH-funded ComPASS projects. As additional partnerships use the two Spanish tools, we will continue to refine them based on psychometrics and feedback.
Finally, the UNM CPR is producing a web app that enables easy implementation with partners, data analysis, and production of a partnership data report for collective reflection. The web app will allow Principal Investigators or project coordinators to input the names and emails of project partners, including academic faculty, staff and students, community researchers, community advisory board members, or other partners. An Excel spreadsheet will provide calculated means for each scale, with the means then inputted into a colorful partnership data report that includes questions to generate partner dialog on their strengths and future strategic actions.
Limitations
The proposed measures have some limitations to consider. There is an inherent challenge in creating universal Spanish instruments due to variations of the language by country, regions, and other contextual factors. While our translation and adaptation process included partners representing different countries and linguistic backgrounds, we encourage users to review with their partners, before use, to ensure the measures address their local context and language needs and that they address concepts or principles that their partnership most cares about. In addition to the translation and back-translation process, an adaptation of the instrument is a key step to ensure a high-quality translation and that the final instrument is one that will be understood by the intended audiences [Reference Erkut30]. Due to the limited number of Spanish-speaking US partnerships that completed the measures during the E2 CES testing in 2020, we were unable to meaningfully analyze their data. However, as previously stated, as additional CEnR and CBPR partnerships use both measures, we will continue to adapt and optimize them.
Conclusion
The Encuesta Comunitaria and the FUERTES measures offer CEnR and CBPR practitioners and partnerships two validated tools for assessing research, practices, and outcomes of their work. They contribute to ongoing national and international efforts, as recently highlighted by the National Academies of Medicine, to develop, test, and disseminate measures of meaningful community engagement and CBPR. Moreover, our development of a short tool, FUERTES, meets the needs of partnerships looking for pragmatic tools to incorporate evaluation into their efforts without adding undue burden to their partners. Updating these measures through reflexive practices will further strengthen CEnR and CBPR involving Latinx communities within the US, Latin America, and globally.
Acknowledgments
In addition to the coauthors of this paper, we thank Dr Laura Chanchien Parajon, then-medical director of AMOS Health and Hope, who guided community health workers to take the Encuesta Comunitaria in Nicaragua. We also thank Dr Leo Morales, University of Washington, for his support in the initial translation of the Encuesta Comunitaria.
Author contributions
Conceptualization: PRE, JP, BB, CD, NW; Data Curation: LESdeL, EAP, BB; Investigation: PRE, JP, BB, CD, MA, SA-G, MA, NW; Methodology: PRE, JP, BB, NW, SA-G; Formal Analysis: JP, BB; Validation: NW; Visualization: PRE, JP; Funding Acquisition: NW; Project Administration: NW; Supervision: NW; Writing-Original Draft: PRE, JP; Writing-Review and editing: all. NW takes responsibility for the manuscript as a whole.
Funding statement
The Spanish-language Community Engagement Survey/Encuesta Comunitaria (CES/EC) and subsequent shorter pragmatic survey (PHIRE/FUERTES) were supported by Engage for Equity: Advancing CBPR Practice Through a Collective Reflection and Measurement Toolkit, with funding from the National Institute of Nursing Research: 1 R01 NR015241 (PI: Nina Wallerstein). We are thankful for our previous Research for Improved Health NARCH V study (U26IHS300293) for the first testing of the CES that led to Engage for Equity with partners (University of New Mexico, University of Washington, and the National Congress of American Indians Policy Research Center). While preparing this manuscript, Dr Rodriguez Espinosa was partly supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR003142, and Dr Peña was supported by the National Institute on Drug Abuse, Award Number T32DA007250.
Competing interests
All authors declare that they have no conflicts of interest. Funding sources did not have a role in the study design; data collection, management, analysis, or interpretation; or writing and approval of this manuscript.
Open access
