To the Editor—A number of hand hygiene monitoring technology (HHMT) options have become commercially available with varying degrees of capability, reflecting a growing market without clear evidence of the impact these systems have on hand hygiene (HH). A systematic review including 42 articles mentioning automated measurement found that fewer than 20% of studies identified calculations for accuracy; among them, the level of rigor was highly variable.Reference Ward, Schweizer, Polgreen, Gupta, Reisinger and Perencevich ¹ Manufacturers’ accuracy assessments are often inadequate and vary with physical location. Given increasing pressure from accreditation bodies to measure HH, ² a methodology is needed to assess HHMT efficacy in a standardized way, allowing for comparison both with the gold standard, direct observation, and with each other. Validation of HHMT must be tested in actual clinical practice to avoid overestimation or underestimation of accuracy.Reference Pineles, Morgan and Limper ³ We propose a rigorous method for validation of HHMT in a medical setting.

HH compliance can be measured in a number of ways. A patient-encounter–based approach measures compliance from just before room entry to just after room exit. This approach requires the observation of the entire patient encounter, which may prove valuable for detection of healthcare worker (HCW) patterns. Alternatively, an independent-event approach treats room entry and exit as separate events, maximizing observer efforts. For HHMT validation, we recommend an extremely independent-event–based approach: each room entry, exit, soap dispenser actuation, and alcohol-based hand rub (ABHR) dispenser actuation should be treated as independent events. This method allows for real-time identification of inaccuracies.

HHMT requires significant investment of resources in proper calibration. Many systems use room “zones” that must be properly calibrated to capture room entries and exits while distinguishing a person walking in a hallway from room activity. In the most sophisticated systems, the interaction between an individual’s identification device and the entry/exit sensor adds additional complexity. Once calibration and structural assessment are deemed acceptable, the basic functionality of the system should be tested using a planned path.

A planned path is a route created throughout a clinical area that allows for systematic validation of HHMT through purposeful activation of every device. In this approach, a planned path is developed for each area, accounting for all soap dispensers, ABHR dispensers, and monitored rooms (Figure 1a). An investigator follows the planned path, activating each device while documenting deviations from the path. Detailed notes are essential for comparison of performed activity with system data. For example, if a room is skipped, it should be noted. Thus, investigators must consider each device activation as a unique encounter.

Figure 1

Comparison of a planned path to natural healthcare provider behavior.

After executing the planned path, data are pulled from the HHMT in the most raw format available. Devices often have a unique identifier, allowing for the association of each event with a specific device or location. Each device encounter recorded by direct observation is compared to data recorded by the system. This approach is valuable for both aggregate and individualized HHMT because it accurately quantifies both activity and attributes of that activity for comparison with performed behavior. For example, a system capable of monitoring individual behavior is assessed for its ability to both document room activity and attribute that activity to the correct individual. There are likely to be inaccuracies identified that require repositioning of devices. After these adjustments, devices can be assessed again using the planned path.

While the planned path quantifies a system’s accuracy in detecting purposeful behavior, behavioral validation quantifies a system’s ability to accurately detect real-world behaviors, which can be chaotic and unpredictable (Figure 1b). Behavioral validation requires trained observers to document all room activity, dispenser actuations, and, for individualized systems, the individual performing each behavior. Observers document unusual behaviors (eg, a patient requiring immediate resuscitative measures) or those that pose difficulty for the technology. For instance, many HHMT observers struggle to accurately distinguish multiple, simultaneous room entries/exits. Thus, when groups enter a patient room, it should be documented.

Once investigators have reached the behavioral validation step of this protocol, basic functionality of the system has been deemed acceptable through the planned path. This allows flexibility in observing natural behavior, with the goal of observing a representative sample of HCWs, locations, and time of day. However, this step should be purposefully planned and systematically executed, ensuring standardization of observations and documentation. Each device activation is again considered a unique encounter with observed behaviors compared to system data.

Accuracy of HHMT can be calculated using fundamental epidemiologic approaches. Sensitivity of HHMT is calculated using direct observation as the accurate data source. This allows identification of true-positive events (those both captured through direct observation and counted by HHMT), false-positive events (events that are not observed but are counted by HHMT), and false-negative events (those which are observed but are not captured by HHMT). Because it would not be feasible to quantify the number of nonevents (ie, how many times a person did not walk into a room or did not perform HH), true-negative events are not captured in this approach. Sensitivity and positive predictive value are then calculated as the measures of system accuracy.

A standardized approach to validating HHMT is the next logical step to establishing standards that allow for comparison of available technologies by the healthcare industry. We recommend the following steps: (1) manufacturers should share the burden of performing validation steps with hospitals; (2) hospitals should require transparency of any accuracy data provided by manufacturers; and (3) either the hospital or a third party should be actively involved in assessing accuracy of HHMT in each physical location where it is installed. The protocol described above provides a method for moving toward these recommendations.