Study design

The study applied an observational, cross-sectional survey design using retrospectively collected EHR data and a quantitative job analysis survey. Based on EHR data, a total of 3498 RNs provided care to 22,703 patients with CHF within 113,543 CHF hospitalizations between January 1, 2015, and January 1, 2024. To test the psychometric properties of a modified Hennessey-Hicks Need Analysis instrument, a sample of 200 RNs were surveyed with respect to the outcomes of their patients. The study was conducted on the main campus of an academic health center in Arkansas, which is a primarily rural state. The hospital participates in the American Heart Association’s GWTG-HF initiative for implementing GDMT. The hospital has 535 beds, of which patients from all 75 counties of the state were provided care, including patients from counties with heart failure-related death rates that are twice the national average [Reference Williams, Crump and Parker15].

Ethical considerations

The study protocol (#276211) was approved by the Institutional Review Board at the University of Arkansas for Medical Sciences. The informed consent process was conducted via email. The document included a basic description of the study purpose, survey completion time, data storage and retrieval processes, and contact information for the principal investigator. A waiver of documentation of the informed consent process was granted by the Institutional Review Board because the study presented only a minimal risk for loss of confidentiality.

Sampling size and methods

A dataset was provided to the study team by the Arkansas Clinical Data Repository. It contained a de-identified list of African American patients with a primary diagnosis of CHF, length of stay, and an edge list containing pseudonyms that linked patients with the RNs who provided their care [Reference Williams, Parker and Bimali14]. Stratified random sampling was used to select 100 RNs from above and 100 RNs from below the mean length of stay of the patients cared for by the total population of RNs. Stratified random sampling was applied to increase the representation of the RNs from the population and the likelihood of selecting RNs from the full distribution of patient length of stay. Based on EHR data, RNs who cared for fewer than 10 African Americans with CHF were excluded to minimize the selection of RNs who lacked sufficient experience in CHF care. Figure 1 shows how the inclusion criteria impacted the sample size.

Figure 1.

Flow diagram of sampling size. RNS = registered nurses; CHF = congestive heart failure.

Instrument development and pretesting

Social, technical, and socio-technical task performance was captured via a modified Hennessey-Hicks Need Analysis instrument. The standard instrument contained 30 items that represent tasks performed during high-quality nursing care delivery [Reference Markaki, Malhotra, Billings and Theus31]. However, the standard instrument was modified because it only had a minor focus on socio-technical tasks, and specificity was added to tailor it to assess clinical information use during the care of African Americans with CHF.

Sittig and Singh’s Socio-technical Framework for Health Information Technology served as the conceptual framework for the study [Reference Williams, Bimali and Garza33–Reference Sittig and Singh34]. The framework is focused on understanding how RNs work with data, information, and technology within its eight interdependent dimensions: 1) hardware and software computing infrastructure, 2) clinical content, 3) human-computer interfaces, 4) people, 5) workflows and communication, 6) internal organizational features, 7) external rules and regulations, and 8) measurement and monitoring [Reference Williams, Bimali and Garza33–Reference Sittig and Singh34]. The framework doesn’t conceptualize nursing care as a simple set of tasks, but as a set of dynamic interactions between people and processes within a complex adaptive system [Reference Williams, Bimali and Garza33–Reference Sittig and Singh34]. The framework was chosen because it was the most comprehensive conceptualization of the social, technical, and socio-technical processes of high-quality nursing care [Reference Williams, Bimali and Garza33].

The modifications of the 30 items were guided by the study’s conceptual framework, and made within the modification requirements described in the Hennessey-Hicks Need Analysis manual [Reference Markaki, Malhotra, Billings and Theus31,Reference Williams, Bimali and Garza33–Reference Sittig and Singh34]. For example, Item #1 of the standard instrument stated: “Establishing a relationship with patients.” The item was modified to state: “Using the EHR to establish a relationship with African American patients who have CHF.” Eight of the items in the standard instrument were removed because they did not align with the role of acute care RNs as articulated by the American Association of Heart Failure Registered Nurses’ Scientific Statement of the Nursing Care of Patients Hospitalized with Heart Failure [Reference Fraser, Barnes and Barsness17]. These modifications were permitted by the manual of the standard Hennessey-Hicks Need Analysis instrument without compromising its psychometric properties [Reference Markaki, Malhotra, Billings and Theus31]. A set of 22 items was retained in the final instrument, now constituting the Needs Analysis for Socio-technical Tasks, which represented three subscales: social, technical, and socio-technical tasks performed by RNs in the provision of CHF care. Of the 22 final items in the Needs Analysis for Socio-technical Tasks, Items 1–14 were modified from the basic items of the standard instrument. Items 15–22 were directly translated from all eight dimensions of Sittig and Singh’s Socio-technical Framework for Health Information Technology [Reference Markaki, Malhotra, Billings and Theus31,Reference Williams, Bimali and Garza33–Reference Sittig and Singh34]. The Needs Analysis for Socio-technical Tasks was also pretested for content validity, usability, and technical functionality in an independent group of five acute care RNs who reflected the sampling frame of the target population of RNs.

Measurement process

A closed survey approach was employed electronically through the Research Electronic Data Capture System between November 15, 2024, and January 31, 2025 [Reference Baker, Bosler, De Fouw, Jones, Harris and Cheng35]. Survey prompts discouraged all RNs from focusing on any specific EHR product, type, or company. Email addresses were used to manage reminders and survey completion. Demographic variables collected using the instrument included biological sex, race, ethnicity, and the number of years working as an RN. The RNs self-rated their performance in all 22 items (tasks) on a seven-point Likert scale (1–7), with higher ratings indicating higher socio-technical performance.

Study variables

The primary study outcome was the length of stay (days). To preserve precision in the outcome measure, it was analyzed as a continuous measure and not rounded to the nearest integer. The study collected two types of independent variables: RN variables and variables on their patients. The primary independent variable was the total socio-technical performance score of the RN. The total socio-technical performance score was computed by summing the RN’s ratings of their own performance in the 22 items on the Needs Analysis for Socio-technical Tasks. Other covariates related to RNs were their years of experience, biological sex, race, and ethnicity. The patient-level covariates included in the analysis were age (years), biological sex, and in-hospital mortality. The health severity of patients was assessed via the Van Walraven Elixhauser Comorbidity Score (ranging from -19 to 89), with higher scores representing poorer health [Reference Williams, Parker and Bimali14–Reference Williams, Crump and Garza16]. The score is a reliable predictor of increased LOS and in-hospital mortality in African American patients with CHF [Reference Williams, Parker and Bimali14–Reference Williams, Crump and Garza16].

Descriptive analysis

Open Refine 2.0 was used to confirm that there was no missing data for any study variables. The data was analyzed using SAS Version 9.4. All continuous variables were analyzed using descriptive statistics. The continuous variables were mean, standard deviation, median, inter-quartile range, minimum values, and maximum values. All categorical variables were presented using frequencies (percentages).

Reliability analysis of the instrument

Classical test theory guided the statistical analysis of determining the extent to which the set of 22 socio-technical items measured the same domain [Reference Williams, Schmidtke, Roessger, Dieffenderfer, Garza and Zozus36–Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. Within classical test theory, item reliability identified items that were able to distinguish between an RN’s task performance and poorly designed items [Reference Williams, Schmidtke, Roessger, Dieffenderfer, Garza and Zozus36–Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. Item reliability was represented by corrected item-total correlations that identified the strength of the relationship between each item and the total socio-technical performance score [Reference DeVellis37]. Corrected item-total correlations below 0.15 are deemed unreliable, and those between 0.15 and 0.30 should be reviewed by experts (bedside RNs) for potential removal [Reference Williams, Schmidtke, Roessger, Dieffenderfer, Garza and Zozus36–Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. The internal consistency reliability of the instrument was estimated using Cronbach’s alpha. The Cronbach’s alpha correlated each of the RN’s ratings of the 22 items with their total socio-technical performance score on the instrument. The 95% confidence interval for Cronbach’s alpha was computed using a non-parametric bootstrapping approach with 10,000 bootstrap samples. Cronbach’s alpha was additionally estimated based on the deletion of each item to assess the impact of removing each item on the instrument’s reliability.

Validity analysis of the instrument

A histogram assessed and visualized the distribution of length of stay. Length of stay was highly skewed and showed a notable departure from normality. Data transformation approaches (i.e., log transformation, square root transformation, and cube root transformation) failed to eliminate the skewness in the data. Therefore, to assess the criterion predictive validity of the Needs Analysis for Socio-technical Tasks, a generalized linear regression with a Gamma probability distribution and log-link function was used to model the association between total socio-technical performance score and length of stay. To account for the within-patient correlation in the study outcome, the Gamma regression models were fitted using generalized estimating equations. Regression modeling was done sequentially to obtain both unadjusted and adjusted measures of association. In step one, the length of stay was regressed on the socio-technical performance score. In step two, the model from step one was adjusted for the following variables: patient age, patient sex, patient Van Walraven Elixhauser Comorbidity Score, RN race, RN sex, and RN years of experience. The change in length of stay was estimated by back-transforming (exponentiating) the regression coefficients from the Gamma regression model. A two-sided P-value of 0.05 was used to determine statistical significance, and uncertainty in parameter estimates was reported using a two-sided 95% confidence interval.

Reliability of the instrument

The Cronbach’s alpha coefficient was 0.95 (95% CI: 0.94–0.96) and remained approximately 0.95 (ranging from 0.947 to 0.953) if any of the 22 items were deleted. This Cronbach’s alpha coefficient fell within the higher range of other published modifications of the instrument (i.e., 0.70–0.95) in areas such as community nursing and cancer care [Reference Markaki, Malhotra, Billings and Theus31]. This indicated a high level of internal consistency reliability within the set of 22 items because it exceeded the established reliability threshold of 0.80 [Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. While a Cronbach’s alpha coefficient of 0.80 to 0.95 is ideal, coefficients above 0.95 may indicate highly correlated and repetitive items [Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. However, the coefficient of 0.95 indicated that the items on the instrument were a well-scoped measure of the same domain. The total set of 22 items was derived from multiple sources: the standard Hennessey-Hicks Need Analysis and Sittig and Singh’s Socio-technical Framework for Health Information Technology. As a result, the items from each of the sources may have been loading more closely together by source due to an artifact of the original measurement design of each source [Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. This may have increased item cohesion and internal consistency by source, subscale, and/or in the overall set of items.

The corrected item-total correlations for each of the 22 items ranged from 0.44 to 0.80. These values indicated that all items were above the established thresholds for reliable items and distinguished an RN’s social, technical, and socio-technical task performance from poorly designed items that were not able to measure their performance (i.e. < 0.15) [Reference Williams, Schmidtke, Roessger, Dieffenderfer, Garza and Zozus36–Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. The results showed that even if the item with the lowest corrected item-total correlation (0.44) was removed from the instrument (Item 13), the Cronbach’s alpha coefficient would have only increased trivially by 0.003 (from 0.95 to 0.953). This indicated that no items should be removed because there was a general latent factor underlying all the items on the instrument [Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38]. Therefore, each item contributed meaningfully to the instrument’s ability to consistently measure the influence of social, technical, and socio-technical task performance. The range of the corrected item-total correlations also indicated that the instrument is multifaceted due to the existence of the three subscales within the instrument: social, technical, and socio-technical tasks [Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38].

Validity of the instrument

While content validity was evaluated during pretesting, the study should be further validated in practicing acute care RNs (i.e., excluding nurse managers without an active caseload) to reduce the potential influence of unidentified confounding variables that may have been unique to the environment of the one hospital that was surveyed in this study. These additional studies of content validity should also employ techniques such as keystroke modeling, task mapping, and task decomposition analyses for each of the 22 tasks. This will give consideration to the complex adaptive systems of nursing care and its seamless interaction with the workflows and multidisciplinary tasks performed by other disciplines in CHF care. These methods will also identify the specific elements of tasks (e.g., time spent performing the tasks, subtasks, variation in approaches) that contribute to optimization and reductions in length of stay.

Criterion predictive validity indicated a high degree of association between the total socio-technical performance score and the length of stay of African American patients (P = 0.004) [Reference Boateng, Neilands, Frongillo, Melgar-Quiñonez and Young38–Reference Salkind39]. The regression found that an increase in total socio-technical performance score of one to four points in each of the 22 socio-technical performance items would result in a decreased length of stay of 3.34% (6.11,0.5), 6.58% (11.84,1), 9.70% (17.22,1.49), and 12.72% (22.28,1.99), respectively. For example, for an RN who cared for an African American patient who had a total length of stay of 3 days (i.e., 72 hours) and whose initial total socio-technical performance score on the instrument was 100, an increase in total socio-technical performance score of 22 points was associated with a 3.34% (i.e., approximately 2 hours and 40 minutes) decrease in the patient’s length of stay. A potential 2-hour and 40-minute decrease in length of stay is clinically meaningful, particularly in hospitals like the one surveyed here, whose capacity is exacerbated by the highest geographic concentrations of CHF prevalence, death rates, and mortality risk that are nearly twice the national average [Reference Williams, Crump and Parker15]. It is anticipated that the specific resulting decrease in length of stay may vary, as model prediction is specific and sensitive to the self-reported performance of the RN. This sensitivity is a strength of the instrument because it addresses national calls to establish a real-time feedback and performance improvement measure of the dynamic nature of nursing care delivery (if embedded and automated within EHRs through existing processes such as the nursing checklists of CHF-specific units) [9]. However, like all data and instruments that have been integrated into clinical workflows, workflow optimization studies should be conducted to ensure the effective translation of the instrument. For example, if integrated into a nursing checklist during the provision of care and RNs are surveyed every shift, it may be simply viewed as just another unimportant task unrelated to their direct provision of care. As a result, response bias could be introduced (e.g., an RN just clicking through the survey to get it done), which would decrease the accuracy of the instrument’s prediction. Workflow optimization studies could address any nuances related to the instrument’s implementation. For example, these studies could assess deployment within EHRs or algorithms to identify scores that are outside the established thresholds for a hospital, unit, or the normal range of a specific RN’s score history (i.e., to detect response bias).

Translation and optimization of guideline-directed medical therapy

The use of the instrument, along with appropriate workflow optimization and training on its use, operationalizes the measurement of clinical information use in GDMT, which is critical to caring for African Americans with CHF. For example, social determinants of health are responsible for 25.8% of the risk for long lengths of stay in African American patients compared to only 10.1% in non-African Americans [Reference Segar, Keshvani, Rao, Fonarow, Das and Pandey7]. While there are many instruments to collect social determinants of health data from patients, they do not measure the effectiveness of the collection process itself, which is critical to ensuring the translation and use of the information [Reference Segar, Keshvani, Rao, Fonarow, Das and Pandey7,Reference Neshan, Padmanaban, Tsilimigras, Obeng-Gyasi, Fareed and Pawlik18]. However, our findings provide a validated instrument for measuring the influence of an RN’s collection of social determinants of health information in nursing care (Item 10).

In the adjusted analysis, the Van Walraven Elixhauser Comorbidity Score (health severity) and in-hospital mortality were significantly associated with length of stay, which is also consistent with recent findings in nursing care [Reference Williams, Parker and Bimali14,Reference Stone, Zwiggelaar, Jones and Mac Parthaláin27]. This finding indicated that the performance of RNs alone is generally not the only influence on length of stay. Therefore, the Needs Analysis for Socio-technical Tasks should be used in conjunction with other instruments and types of data used in CHF care (i.e., ranging from anatomical and physiological data to healthcare system data) to comprehensively measure, stratify, predict, and control for all influences on length of stay [9,Reference Kao, Trinkley and Lin13,Reference Neshan, Padmanaban, Tsilimigras, Obeng-Gyasi, Fareed and Pawlik18–Reference Lake, Rosenbaum, Sauveur, Buren and Cho29]. The instrument’s strong validity and reliability present an opportunity to establish the 22 items as national quality measures of nursing care within initiatives such as the American Heart Association’s GWTG-HF [Reference Tang, Lewsey and Yancy8,9,Reference Williams, Parker and Bimali14,Reference Stone, Zwiggelaar, Jones and Mac Parthaláin27]. Subsequently, structured interventions and toolkits should be developed to address and increase the performance of RNs, by task, which could reduce the lengths of stay of patients. For example, identifying and increasing performance in Item 10 (i.e., the collection of social determinants of health information) could decrease lengths of stay through interventions such as providing transportation vouchers for follow-up visits to a cardiologist, optimizing patients’ receipt of GDMT [Reference Segar, Keshvani, Rao, Fonarow, Das and Pandey7]. The instrument also measures information use in other areas unique to chronic disease care. These areas include assessing the use of patient data collected longitudinally, as the top 5% of patients with multiple chronic diseases have historically accounted for 50% of healthcare spending in the United States (Items 15,17,20,21,22), patient self-management (5 and 8), and assessing the influence of care delivery (Items 3,4,12,13,14) [Reference Driscoll, Meagher and Kennedy11,Reference Fraser, Barnes and Barsness17,Reference Stone, Zwiggelaar, Jones and Mac Parthaláin27,Reference Mitchell40,Reference Williams, Garza and Lipchitz41]. The instrument could improve prescribing within GDMT by assessing the use of longitudinal data related to searching for and importing clinical data into notes, visualizing trends in laboratory values and previous medications, and triggering clinical recommendations [Reference Kao, Trinkley and Lin13]. For example, the instrument could assess an RN’s use of a structured, CHF-specific note that was prepopulated with trends in laboratory values [Reference Kao, Trinkley and Lin13]. For hospitals whose GDMT is formally guided by GWTG-HF, the instrument further facilitates the use of information related to: the evaluation of heart function (Items 9 and 16), using information based on evidenced-based care (Items 2 and 7), structured discharge CHF education for self-care (Items 5 and 8), and care communication and coordination processes such as scheduling follow-up visits within 72 hours of discharge (Items 1,11,18,19) [Reference Driscoll, Meagher and Kennedy11].

Limitations

The study is based on a single academic health center in a rural state and may not reflect the structure of nursing care delivery at all hospitals in the United States. This may limit the generalizability of the findings and the instrument to only RNs, African American patients, and hospitals in the United States with similar characteristics as the RNs, African American patients, and hospitals presented in this study. Additionally, the Van Walraven Elixhauser Comorbidity Score was used as the sole measure of patient health severity. There are many additional indicators of health severity, including left ventricular ejection fraction, that may be appropriately applied to strengthen the representation of a patient’s health severity. Based on the study design, the instrument provides a nursing-based self-assessment of their own performance, which could be perceived as subjective in nature. As a result, the instrument should be used in collaboration with established objective measures of care in GDMT, such as defect-free care for quadruple therapy for patients with reduced ejection fraction [Reference Tang, Lewsey and Yancy8,9]. Further research is needed to determine the specific proportion of weight that should be placed on subjective and objective measures within clinical risk stratification processes and adjustments to data analyses. This will ensure the effective accounting of and cohesiveness with other factors, such as heart function, that may more directly influence length of stay.