Since the first generation of digital communications, electronic communication devices have become more affordable and increasingly sophisticated, leading to nearly ubiquitous use of portable Internet-connected devices in society( Reference Armstrong, Giovinco and Mills 1 ). Consequently, these technologies have increasingly been integrated into health care at a number of levels with positive results( Reference Free, Phillips and Felix 2 ). Hospitals and medical schools now commonly use remote audio- and video-based technologies for instruction and training( Reference Hayden, Navedo and Gordon 3 – Reference Burckett-St Laurent, Cunningham and Abbas 5 ), and many physicians provide video consultations through real-time online consultation platforms such as MDLiveCare and SwiftMD( Reference Glauser 6 ). Health care providers are even using digital real-time video technology platforms to deliver health care in rural and/or resource-limited areas( Reference Miyashita, Iketani and Nagamine 7 – Reference Daugherty, Schap and Ettienne-Gittens 12 ) and to remotely diagnose, monitor or treat medical conditions ranging from orthopaedic trauma to neurological disorders( Reference Armstrong, Giovinco and Mills 1 , Reference Armfield, Bradford and Bradford 10 , Reference Good, Lui and Leonard 13 – Reference Boots, Singh and Terblanche 21 ).
Despite the rapidly growing use of telehealth in clinical care, very few dietary, lifestyle or pharmaceutical research studies use visual digital communication technologies as a clinical research tool. A small number of research studies have used video chatting platforms such as Skype and Facetime to conduct interviews, thereby supplanting telephone or in-person interviews( Reference Weinmann, Thomas and Brilmayer 22 – Reference Janghorban, Latifnejad Roudsari and Taghipour 24 ). Only a few clinical research studies have used visual electronic technologies to capture novel information, such as to collect health behaviour data in real-world settings or to more accurately quantify intervention adherence( Reference Park, Howie-Esquivel and Dracup 25 ), which is the ability of participants to follow prescribed medication or lifestyle changes. An example of the novel application of visual communication technology to dietary research is food ‘photography’ methods such as the Remote Food Photography Method© (RFPM) and SmartIntakeTM smartphone application( Reference Daugherty, Schap and Ettienne-Gittens 12 , Reference Martin, Correa and Han 26 – Reference Ptomey, Willis and Honas 28 ). Using a smartphone app, study participants take still images of the food they eat, and the smartphone images are relayed back to researchers to estimate the quantity and quality of food consumed. This technique has been found to accurately measure the energy and nutrient intake of adults( Reference Daugherty, Schap and Ettienne-Gittens 12 , Reference Martin, Correa and Han 26 – Reference Ptomey, Willis and Honas 28 ), and it can be used both for real-world studies of health behaviours and for monitoring of adherence to controlled feeding interventions.
For the latter purpose – to monitor dietary adherence – RFPM is frequently thought to be better than data collection and monitoring methods such as self-report, which is widely known to be of limited accuracy( Reference Beaton, Burema and Ritenbaugh 29 – Reference Bandini, Schoeller and Cyr 32 ). For monitoring adherence to other health behaviours – such as supplement or medication taking – visual monitoring technology has not yet been implemented, although novel technologies such as the Medication Event Monitoring System (MEMS; AARDEX Ltd) can accurately quantify when pill bottles are opened via a computer chip that is built into a bottle cap. MEMS is marginally better than standard pill counts and even unannounced telephone-based pill counts( Reference Fredericksen, Feldman and Brown 33 ), yet none of these methods can detect whether pills were removed from the container and discarded/not consumed.
By contrast, video chat technology has the potential to solve the problem of monitoring dietary and pill-taking (i.e. supplement and medication) adherence by providing video of the behaviour from start to finish, minimising the ability of participants to ‘cheat’ or engage in non-adherent behaviours. Although there are a multitude of studies investigating the feasibility and usability of video chat technology to assess health behaviours, currently a small number of clinical research studies have used the technology to monitor or enhance adherence to a clinical intervention( Reference Beaton, Burema and Ritenbaugh 29 , Reference Noda, Sakata and Kubota 34 – Reference Sakata, Kubota and Uemura 47 ), and none have tested its validity and reliability for this purpose. We therefore designed a pilot study to empirically evaluate the validity and reliability of video chat technology to quantify adherence to dietary and pill-taking interventions remotely compared with the gold standard of in-person monitoring. We hypothesised that diet and pill adherence could be quantified as reliably and accurately by webcam as in-person. In parallel, we also administered a survey to determine the acceptability and feasibility of using video chat technology to participate in clinical research. To our knowledge, this is the first pair of studies to rigorously investigate the issues of the validity, reliability, acceptability and accessibility of video chat technology in the context of clinical intervention adherence.
Methods
Both studies were conducted at Pennington Biomedical Research Center, approved by the institution’s Institutional Review Board and registered in the clinicaltrials.gov registry (www.clinicaltrials.gov; NCT no. 02204540). The studies were conducted in accord with the Declaration of Helsinki, and no compensation was provided for either study.
Validity and reliability study
The aim of the first study was to assess the accuracy and reliability of detecting compliance to dietary and pill-taking interventions by video chat, in comparison with in-person monitoring. To test validity, seven research centre staff members were recruited (employee status was the only inclusion/exclusion criterion), and all seven provided written informed consent. Four of the seven volunteers served as monitors, and the remaining three served as mock study participants while: (1) eating meals, as if they were participating in a dietary clinical study; and (2) swallowing pills, mimicking participants in a supplement or pharmaceutical trial. The participants ate the meals and swallowed pills (Biotin vitamin capsules were used for this trial), while following behavioural scripts that outlined instructions to perform non-adherent behaviours (called ‘cheats’).
Cheating behaviours
Non-adherent behaviours include deviant actions such as not eating all of the food by spitting food into a napkin, placing a pill under one’s tongue and discarding it and so on. We developed a list of common cheating behaviours by convening a group of six registered dietitians and staff members from Pennington Biomedical Research Center who professionally monitor study participants for compliance in clinical studies. For reference, the focus group estimated that <5–10 % of study participants blatantly cheat, and that the most common reason for cheating is study fatigue. The cheating behaviours identified as common during dietary interventions included hiding food in a napkin and not eating it; stacking containers to hide unfinished food; and removing some of the food from a container before the meal monitoring starts.
The mock study participants were given standardised instructions for eating and swallowing the pills (online Supplementary Material S1). These instructions were designed to be identical to those that would be given to real study participants to make it easier for monitors to accurately assess compliance. The instructions included showing the pill to the camera; showing the participant’s empty hands to the camera after swallowing the pill; showing each empty food container to the camera after eating; and keeping their hands and head in the field-of-view of the webcam at all times. In addition, when scripted to cheat, participants were instructed to deliberately avoid being caught cheating.
The three mock study participants together ate a combined total of thirty meals (192 food items), during which sixty cheating events were scripted to occur, and they swallowed a total of sixty pills with thirty scripted cheating events. For simplicity, the pill-swallowing tests were performed in conjunction with the dietary adherence tests: one pill was swallowed before each meal and a second pill was swallowed after each meal. (Because of the unforeseen circumstances for one of the mock study participants, half of the cheating behaviours for both meals and pills was performed by one participant, whereas the other half of the cheating behaviours was unevenly divided among the other two mock participants.) The cheating events were scripted as follows. Each meal was randomised to contain an average of two cheating behaviours (range 0–4), and each instance of pill swallowing was randomised to have 0 or 1 cheating behaviours. The specific cheating behaviour that was performed in each instance was also randomised. The monitors were blinded as to the total number of cheats. On rare occasions, participants accidentally forgot to cheat as instructed or performed additional cheats; thus, the results are expressed as a percentage of all behaviours that actually occurred.
Monitoring
During data collection, two monitors watched each meal or pill swallow in-person and two additional evaluators rated each event remotely by watching the video-chat-recorded videos. The monitoring was conducted in a cross-over design, so that each pair of monitors observed both in-person and by video chat in a balanced order. The monitors were employees of the research centre with some exposure to either RFPM or monitoring of meals but were not trained research dietitians and did not have intensive training, nor were they trained for this study, in monitoring compliance for dietary or pill-taking intervention trials. Similar to actual study procedures, monitors were given images of the actual meals to be consumed and the pill to be swallowed about 5–10 min ahead of time, and they documented any cheating behaviours on a standardised form. In-person monitors were seated 1·5–1·8 m away from the participant. A large divider was placed between the two observers, and each wore earplugs, so that they could not see or hear each other. To video record each instance of eating a meal or swallowing a pill, a smartphone was placed about 1–1·5 m away from the participant. All recordings were performed using Skype and CallNote premium on either a Samsung smartphone or an iPhone, paired with both a tripod and wide-angle lens. We describe in detail how we identified this combination of software and hardware as being optimal in the online Supplementary Material S2, and we strongly encourage clinicians and researchers who want to implement video chat technology to read this section for advice. The video recordings were made by smartphone, as more people have smartphones than other devices with webcams (see the ‘Results’ section), but our methods were designed to generalise to video chat technology in general. Last, an exit survey was given to monitors to gather feedback on their experiences with monitoring compliance by video chat technology, to determine what they liked and did not like about monitoring by video chat and to determine what technical aspects of the video quality made it easier or harder to detect cheating.
Statistical analysis
The validity (i.e. accuracy) of detecting cheating by video chat in comparison with in-person was quantified by sensitivity (percentage of cheats that were detected) and specificity (1 – false positive rate, where each pill or food item consumed without cheating was counted as one event). The motivation for this manuscript was the dietary adherence sensitivity testing, which was powered at the 80 % level to detect a 15 % absolute difference between in-person and video chat monitoring, assuming 95 % of cheats were detected in-person (one-sided test). The pill-taking testing was added later as an exploratory pilot analysis, as we could find no reasonable published or anecdotal data to formulate an estimate for the percentage of cheats that could be detected in-person. Reliability was operationally defined as inter-rater agreement adjusted for agreement by random chance, which was quantified primarily by Cohen’s κ; Cronbach’s α was also calculated as a second measure of inter-rater agreement. Statistical differences between in-person and video monitoring of compliance were assessed using Fisher’s exact test. The type I error rate (α) was set at 0·05 for all analyses.
Feasibility and acceptability study
To investigate the feasibility and acceptability of using video chat technology to participate in clinical research, a twenty-two-question survey that takes <5 min to complete was developed (online Supplementary Material S3).
Survey respondents
The survey was administered through SurveyMonkey (www.surveymonkey.com) and was sent via a listserv that Pennington Biomedical Research Center uses to promote its clinical trials. The listserv contains more than 20 000 unique email addresses, although it is unknown how many of those are currently valid. Survey responses were collected primarily over the 2-month period from December to January 2014. Because study 2 included an anonymous online survey, a waiver of informed consent was granted. There was no testing for legitimacy of email addresses or for validity of content.
Survey design
We designed a close-ended survey to investigate whether study participants have access to, and experience with, video chat technology (feasibility), and whether they are willing to use the technology in clinical research (acceptability). In addition, as people often cite scheduling conflicts and commuting time as reasons for not participating in clinical research, we also investigated whether video chat technology might reduce barriers to participating in clinical research in general. Section 1 of the questionnaire asked respondents about their past participation in clinical studies, whether scheduling difficulties prevented them from participating in studies and whether offering study visits on evenings or weekends would help them participate in more studies. Section 2 asked respondents to indicate their comfort with, and prior use of, video chatting, what video chatting software they have used, their access to webcams for video chatting (home and/or work) and what hardware they own (e.g. smartphone). Section 3 asked participants about whether they prefer to do study visits by video chat or in-person clinic visits (along with the reasons why), whether they wanted behavioural support by video chat and whether they would be willing to use video chat technology to participate in a clinical trial. Section 4 included demographic questions.
Statistical analysis
Survey responses are expressed as a percentage of those who answered each question. χ 2 Tests were performed to test whether survey responses differed by demographic variables. Given the multitude of association tests, the Bonferroni correction was applied. All statistical tests performed were two-tailed.
Results
Validity and reliability study
Dietary adherence monitoring
As shown in Table 1, inter-rater agreement by Cohen’s κ for dietary adherence monitoring was high for monitoring both in-person and by video chat, at 0·88 and 0·85 (P=0·62), respectively. This was supported by values of 0·94 and 0·92, respectively, for Cronbach’s α. The sensitivity (true positive rate) for detecting cheating in-person was 86 %; surprisingly, in-person monitors did not detect about one of six cheats. The sensitivity of detecting cheating remotely through video chat was 78 %, which was not significantly different from in-person monitoring (P=0·12). When examining individual cheating behaviours (Table 1), there were no statistically significant differences between monitoring in-person v. by video chat (P values>0·10). Removing food before monitoring started was the most common cheating behaviour not detected by the monitors, with the behaviour detected <20 % of the time. Spitting food into a napkin was also difficult for raters to detect, with only about half of the events detected by in-person and remote monitors. Monitors detected the remaining cheating behaviours most of the time. For both the in-person and remote monitoring, the false positive rate was very low (1 %).
Sensitivity, specificity and inter-rater agreement for monitoring dietary and pill-taking adherence and percentages of specific cheating behaviours detectedFootnote †
* P<0·10, ** P<0·05.
† ‘Pill was empty’ means that the powered contents with the active ingredient were removed from the clear pill capsule, whereas ‘No pill was present’ means that participant pretended to clutch a pill and to swallow it, but no pill was ever there.
Pill-taking adherence monitoring
Although the inter-rater agreement by Cohen’s κ for in-person monitoring of pill-taking adherence (0·85) was comparable to that for dietary adherence, inter-rater agreement was somewhat lower for monitoring remotely through video chat (0·69); however, this difference did not reach statistical significance (P=0·21). Similarly, Cronbach’s α was 0·93 for in-person inter-rater agreements v. 0·82 for video chat monitoring. The sensitivity of detecting cheating was 77 % in in-person monitoring, meaning that about one-quarter of cheating events were not detected by the gold-standard method of in-person monitoring. By comparison, the sensitivity tended to be lower at 60 % for remote monitoring through video chat (P=0·08). The cheating behaviour least likely to be detected was hiding the pill in the mouth, such as under the tongue, instead of swallowing it. In-person monitors detected this behaviour correctly 50 % of the time, whereas the video chat monitors did not detect it (0·0 %; P=0·08). In addition, spitting the pill in a drinking glass instead of swallowing it was easy to detect in-person (100 %), whereas monitors detected this behaviour significantly less frequently when they were monitoring by video chat (38 %; P=0·03). Similar to monitoring of dietary adherence, the false positive rates were very low (0·0 and 2 %; P=0·50) for both in-person and remote monitoring, respectively.
Acceptability by monitors
Three of the four monitors were somewhat more confident in their ability to detect cheating in-person v. recorded videos, with one monitor finding remote monitoring by video chat easier. Most monitors reported that the video resolution and lighting were the two most important factors that affected their ability to detect cheating behaviours in the recorded videos. In the case of lighting, glare and identifying the correct food item/pill were the most common issues. In addition, it was clear that the frame rate (number of frames per second) of the video influenced the ability to detect cheating for the pill adherence tests. Frame rate and resolution are largely determined by the Internet connection speed. As it happened, these varied in the area of the building where the validity and reliability tests were conducted, allowing us to qualitatively assess their impact on sensitivity. When the frame rate and resolution were low, it was easier for participants to drop the pill or hide the pill without the monitor detecting it, thus reducing the sensitivity of detecting cheating remotely, whereas the sensitivity of monitoring dietary compliance was less affected by video quality. All monitors reported that being able to pause, rewind and fast-forward the videos was very or extremely helpful in detecting cheating.
Survey respondents
For the second study – which investigated the feasibility and acceptability of video chat technology – 1076 respondents completed the online survey. The respondents’ demographics are shown in Table 2. Approximately three-quarters of participants were female. Nearly 70 % of respondents were Caucasian, and one-quarter were African or African-American, consistent with local demographics. Approximately half of the respondents had participated in one or more clinical studies in the past, and all were interested in participating in clinical research in general. As shown in Table 3, 48·0 % of respondents reported that scheduling conflicts prevented them from participating in clinical research on at least one occasion. Importantly, approximately three-quarters of the respondents reported that being able to schedule study visits on evenings and/or weekends would enable them to participate in more research studies, with 32·6 % of respondents preferring having the option of both evening and weekend visits. Transportation issues were less of a barrier to study participation: nearly three-quarters of respondents reported that transportation did not prevent them from participating in clinical research (Table 3). Nonetheless, one-fifth (21·3 %) reported that a long commute time/bad traffic made it difficult for them to be involved in clinical research. These data demonstrate that video chat technology may also be useful to supplant in-person clinic visits and to reduce barriers to participating in clinical research in general.
Demographics of survey respondents (n 1076)
Scheduling conflicts are a barrier to study participation and technology use and comfort levels among survey respondents (n 1076)
* A question that allowed multiple responses.
Feasibility of video chat technology
Next, we investigated whether video chat technology might be a feasible tool for use in clinical research. As shown in Table 3, 86·4 % of respondents reported that they have the hardware necessary for video chatting. As 81·8 % reported owning a smartphone with a webcam, a webcam-endowed smartphone was clearly the most widely accessible video chat platform. In terms of comfort with video chat technology, 73·3 % of respondents are very comfortable or comfortable with using video chat technology, whereas only 7·1 % are uncomfortable or very uncomfortable (Fig. 1). As shown in Table 3, this figure is higher than that for comfort with technology in general, for which only about 40 % of respondents reported being very comfortable or comfortable. Importantly, nearly 80 % of respondents have participated in live video chatting before, with Skype (68·0 %) and FaceTime (60·7 %) being the two most popular video chat software programs. Google Hangouts was in distant third place at 13·7 % usage (Table 4).
Most respondents embrace the use of video chatting in research studies. (a) 73·3 % of respondents were comfortable with video chat technology, whereas only 7·1 % were not. 
, Very comfortable; 
, comfortable; 
, neutral; 
, uncomfortable; 
, very comfortable. (b) Almost twice as many respondents preferred study visits to be done remotely via video chat (45·2 %) v. in-person (25·3 %). , Prefer video chat; , no preference; , prefer in-person visit. (c) A majority of respondents were interested in receiving behavioural support to help them adhere to study interventions via video chat. , Yes; , maybe; , No. (d) Nearly 80 % of respondents were willing to use video chat to participate in research studies, whereas only 5·4 % were opposed. About a quarter of those willing to use video chat would agree to using the software only on the condition that the video chat session was not recorded. , Yes; , maybe; , No.
Respondent reasons for preferring in-person study visit v. remote study visits via video chatFootnote †
† Both questions permitted multiple responses.
Acceptability of the technology
Finally, we queried participants to determine whether they would be willing to use video chat technology to participate in research. When offered an option to conduct a study visit via video chat or in-person, nearly half (45·2 %) preferred to conduct the visit remotely by webcam, with only 25·3 % preferring to have the visit in-person (Fig. 1); the remaining had no preference. Of those preferring an in-person clinic visit (Table 4), the most common two reasons were having in-person contact/accountability (68·6 %) and disliking being watched by video chat (39·3 %); discomfort with or lack of the technology were minor contributing reasons (17·0 %). Of those who instead prefer doing remote visits via video chat, most cited the commute (62·6 %) and their work (59·6 %) and family/social (44·2 %) schedules as the reasons why. Additional reasons included liking using technology (39·5 %), living or working too far away (35·0 %) and preferring to save money on transportation (33·4 %). Fig. 1 shows that if offered an option of receiving behavioural support to adhere to the study intervention (i.e. encouragement and motivation to stick to the intervention) by video chat, 57·7 % would want it and another 33·3 % declared they might want it. Last and most importantly, nearly 80 % of respondents were willing to use video chat technology to participate in a clinical trial, whereas only 5·4 % were opposed. About one-quarter of those willing to use video chat technology would agree to using the software only if video chat session was not recorded (data not shown).
Demographic associations
We tested the survey results for demographic associations. There were no associations between sex and the survey questions. There was one association with ethnicity – Hispanics were less comfortable with using technology in general (P=0·0009) – although the number of Hispanics in our sample was small (n 22), and thus we caution extrapolation of this one result. To test for associations with race, we limited testing to Caucasians and African-Americans because of low numbers of respondents in other racial groups. In comparison with Caucasians, African-Americans were more likely to report that scheduling conflicts prevent them from participating in research (73·5 v. 60·2 %; P=0·0001) and that being allowed to schedule study visits on weekends would help them participate in more studies (63·2 v. 52·5 %; P<0·0001). African-Americans also were more likely to want behavioural support through video chatting than Caucasians (69·3 v. 54·6 %; P=0·0002). Interestingly, African-Americans were less likely to have a computer with Internet access and a webcam (P=0·0001), but fortunately just as likely as Caucasians to have a smartphone equipped for video chatting (P=0·46).
Age was associated with the responses for nearly all survey questions; however, a majority of associations were modest in effect size (i.e. <25 percentage-point difference in survey responses between the very oldest and youngest cohorts). Older respondents were less likely to report that transportation or scheduling conflicts prevented them from participating in studies (P<0·0001); slightly less likely to have a webcam-enabled device for video chatting (P<0·0001); less comfortable with technology in general and with video chatting (P<0·0001); and less inclined towards using (P<0·0001) and less willing (P=0·0003) to use video chat technology to participate in clinical trials. In particular, 90 % of respondents aged 18–24 years old were willing to use video chat technology to participate in clinical trials, whereas 62 % of those aged 65+ years old were willing to use the technology. Similar numbers – 99 and 62 % of those aged 18–24 and 65+ years old, respectively – had prior experience in video chatting. The one exception to these trends with increasing age is that respondents in the middle two age groups (35–44 and 45–55 years old) were somewhat more likely to want motivational support by video chat compared with either their younger or older counterparts (P=0·0001).
Discussion
Current methods to quantify dietary and pill-taking adherence in free-living participants rely on strategies known to be of limited accuracy, such as self-report, pill counts and returning empty food containers. However, circumventing these issues by conducting inpatient or highly supervised studies – such as in controlled feeding studies – to ensure participant adherence also has disadvantages, namely high participant burden, high staffing needs and high costs. New strategies to measure and encourage intervention adherence in clinical trials are therefore needed.
In this study, we propose video chat technology as a novel method to address both these problems. In free-living studies, video chat technology can be used to remotely quantify adherence, thereby replacing self-report, pill counts (for pharmaceutical and supplement studies) and the ‘empty container’ method for estimating dietary adherence (such as counting yogurt lids in a probiotic yogurt study). Not only could video chat technology improve adherence estimates, but it could also potentially reduce participant burden and encourage better adherence. In controlled studies that might otherwise require inpatient stays or additional trips to the research site, video chat technology could alleviate the need to commute or stay continually at the research centre to ensure compliance. For instance, in a meal timing study, participants could demonstrate that they followed the assigned eating schedule by logging onto video chat software at home or at work and eating meals at the appropriate times. Finally, video chat technology could even be used in epidemiological studies to replace food diaries by capturing data on the type of food eaten, the time of day and even estimated portion sizes.
However, whether the technology is effective and feasible to use in clinical research is an open question. Therefore, in this pair of studies, we investigated the validity, reliability, feasibility and acceptability of using video chat technology to quantify dietary and pill-taking (supplement and medication) adherence. The first study was a pilot study designed to test the ability of video chat technology to detect non-adherence to dietary and pill-taking interventions, in comparison with the gold standard of in-person monitoring – the first time such a study has been performed. For dietary adherence monitoring, the reliability or agreement among raters calculated using Cohen’s κ for both in-person and video chat monitoring was excellent (0·88 v. 0·85), with no statistical difference between the two methods. The validity testing revealed that 86 and 78 % of cheats were detected by in-person and remote meal monitoring, respectively, and the methods did not differ from each other. Interestingly, we found that the mock study participants could successfully evade being caught cheating one in six times, even by the gold standard of in-person monitoring. One cheating method that was very difficult to detect using either monitoring method was the removal of food from the container before beginning monitoring. To address this limitation, meal monitors may need to be better trained (our volunteers were untrained staff members) or the way in which the foods are packaged may need to be modified to better indicate whether the package was opened before the monitoring period. In addition, spitting food into napkins was also difficult to detect by either method, although this limitation can be addressed by requiring study participants to shake out their napkin after the meal. Overall, video chat technology proved to be comparably valid and reliable to in-person monitoring of dietary adherence, and we therefore feel confident that it is ready for adoption as a widespread method for quantifying adherence in nutrition research, particularly for controlled feeding studies.
However, our exploratory testing of pill adherence found that monitoring remotely by video chat trended towards being inferior compared with in-person monitoring. Inter-rater agreement by Cohen’s κ was higher for in-person monitoring (0·85) than video chat (0·69) monitoring. Unlike meal monitoring, detecting non-adherence when taking pills tended to be more difficult via video chat (60 % of cheats detected) compared with in-person monitoring (77 %), yet only two cheating behaviours were or tended to be significantly harder to detect by video chat (hiding the pill in the mouth and spitting the pill into the cup without swallowing it). The fact that these differences trended towards significance is likely because of the fact that our statistical power was more limited. Although our pilot study for pill adherence was not statistically powered, the detection of only 77 % of cheats in-person would have weakened post hoc statistical power by almost a factor of two if the study has been powered identically to the dietary adherence testing; thus, it is appears that about double the number of non-adherent events as in our dietary adherence testing is needed for future full-scale validity and reliability testing of pill-taking monitoring.
In addition, detection of some cheating behaviours remained difficult for either method, such as hiding pills under the tongue. Closer examination of the recorded videos revealed that natural fluctuations in Internet speed and lighting sometimes reduced the resolution and frame rate of the video, making it easier for the participants to avoid getting caught cheating during pill taking. This provides some insight into ways in which the ability to detect cheating during pill taking via video chat can be improved. A faster Internet connection, good lighting and higher-resolution video may increase the sensitivity of monitoring pill adherence by video chat, and such technical resources are expected to be more widely available over time.
On the basis of our pilot study of pill adherence monitoring, further optimisation of the technical setup and a larger sample size follow-up study are needed to determine whether video monitoring of pill-taking compliance is truly inferior to in-person monitoring. Unlike monitoring dietary adherence, we suspect that monitoring pill-taking adherence by video chat technology will likely prove to be inferior to in-person monitoring, although this needs to be confirmed in a larger study. Despite these limitations, the platform is clearly superior to no monitoring at all and to self-reported adherence, pill counts and inspecting empty pill containers. These methods allow participants to discard pills and foods surreptitiously, while falsely pretending to have been compliant. Furthermore, even if the validity and reliability are slightly inferior to in-person monitoring for pill-taking adherence, it is important to remember that in-person monitoring is rarely feasible because of the high burden (e.g. scheduling constraints and commute time) it imposes on participants. Moreover, several controlled studies have demonstrated that simply monitoring pill-taking adherence remotely by video can increase adherence rates( Reference Noda, Sakata and Kubota 34 , Reference Hommel, Hente and Herzer 36 , Reference Hommel, Gray and Hente 39 , Reference Smith, Lunde and Hathaway 40 , Reference Carroll and Trappe 45 , Reference Skrajner, Camp and Haberman 46 , Reference Haimoff and Rudin 48 , Reference Saberi, Yuan and John 49 ), relative to no form of visual monitoring or self-report and it boosts adherence rates to levels similar to those achieved with in-person monitoring( Reference Johnston, Wheeler and Deuser 15 , Reference Harris, Freeman and Duke 35 ). Moreover, when no video is received from a participant by a certain time each day, the participant’s non-adherence can be detected in real time, and s/he can then be reminded to take the pill or follow his/her prescribed dietary intervention( Reference Sakata, Kubota and Uemura 47 ). Therefore, this study provides evidence that video chat technology provides a valid and reliable platform for remotely quantifying diet and pill adherence and likely also for encouraging better adherence.
The second study investigated the feasibility and acceptability of using video chat technology to participate in clinical research – the largest study to-date to test the feasibility and acceptability of using webcam or video chat technology either in clinical research or in patient care (>1000 participants) – with the important finding that it is a feasible and acceptable method for the overwhelming majority of potential study participants. A majority (86·4 %) of respondents had the hardware necessary to video chat, with nearly the same proportion having a smartphone configured for video chatting to occur at any location with either a WiFi Internet connection or a cellular signal. In addition, approximately three-fourth of respondents were familiar with video chat and nearly half (45·2 %) preferred to conduct study visits via webcam or video chat. Finally, nearly 80 % of participants were willing to use video chat technology to participate in clinical research. This concords with several clinical care studies conducted with small numbers of patients that have similarly demonstrated very high satisfaction (range 65–93 %) and good feasibility with remote video monitoring( Reference Armfield, Bradford and Bradford 10 , Reference Good, Lui and Leonard 13 , Reference Johnston, Wheeler and Deuser 15 , Reference Chan, Callahan and Sheets 44 , Reference Robinson, Branham and Locklear 50 , Reference Antonini, Raj and Oberjohn 51 ). Collectively, these data indicate that few technological barriers exist to conducting study visits via video chat, participants are familiar with and accepting of the technology and that in fact more respondents preferred to conduct study visits via video chat compared with in-person.
Although not the focus our investigation, the survey also revealed that approximately one-half of former and potential study participants encountered scheduling difficulties that prevented them from participating in one or more research studies – a problem that may also be mitigated by video chat technology. Importantly, the timing of clinic visits within people’s busy schedules, not transportation, was the primarily issue for most individuals. Video chat technology offers a novel alternative that can reduce these barriers. Conducting study visits remotely by video chat reduces the inconvenience and expense of visiting the clinic. More specifically, it can allow appointments to be scheduled in between commitments that are otherwise too close together to commute to the clinic, to be scheduled outside of business hours or even to happen spontaneously at the participant’s convenience. The method particularly has value for controlled feeding studies, where the high participant burden has made such studies increasingly difficult to conduct and has resulted in fewer controlled feeding studies. A related but unanticipated finding is that video chat technology may particularly help reduce barriers to research participation experienced by African-Americans, a group that is frequently under-represented in research studies( Reference Shavers-Hornaday, Lynch and Burmeister 52 ). African-Americans were more likely than Caucasians to report that scheduling conflicts prevented them from engaging in research and to want behavioural support through video chatting.
Our two studies are not without limitations. In our validity and reliability study, the pill-taking tests were determined to be underpowered in a post hoc analysis, although they do provide the first data that can be used to power studies on pill adherence. In retrospect, it would have been better to provide more rigorous adherence training to the monitors, as monitors did not always agree on whether a food item was fully eaten if, for example, some sauce was left on the plate. In our feasibility and acceptability study, the main limitation is that about three-quarters of our sample were women and all respondents had to have email addresses. As a result, our study sample was likely enriched in female users who are more technology-savvy than average, which limits generalisability. Nonetheless, we found no sex differences in response to any of the twenty-two questions. Furthermore, we recognise that there are notable limitations to using video chat technology to monitor adherence or conduct visits remotely that were not addressed by our studies. Most obviously, study visits involving blood draws for bioactive compounds cannot be conducted remotely. In addition, study participants must have a data plan or requisite Internet access and must be in proximity of their webcam-enabled devices if their study appointment is scheduled at a particular time.
In summary, the two studies reported herein were, to our knowledge, the first of their kind to determine whether video chat technology is an acceptable and feasible method to participate in dietary or pharmaceutical and supplement clinical research, and to empirically evaluate the ability of video chat technology to remotely quantify adherence. Validity and reliability by video chat were excellent for dietary adherence and were decent but tended to be inferior to in-person monitoring for pill adherence. About 80 % of participants have the technology, and a similar percentage were willing to use the technology to participate in clinical research. It is therefore expected that video chat technology will be increasingly used to monitor dietary adherence, to collect study data and even to conduct study visits remotely in order to reduce participant burden and barriers to participating in research. Video chat technology is therefore a very promising tool that is ripe for integration into clinical research methods.
Acknowledgements
The authors are grateful to their study participants, without whom this research would not be possible. The authors also thank Jonathan Goldman and Dr Josh Boehm for their advice on video chat technology.
This work was supported by a NORC Center Grant P30 DK072476 entitled ‘Nutritional Programming: Environmental and Molecular Interactions’; by 1 U54 GM104940 (to C. M. P. and J. W. A.) from the National Institute of General Medical Sciences of the National Institutes of Health, which funds the Louisiana Clinical and Translation Science Center; and by the Albert Noyce Summer Internship Program (to C. W.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
C. M. P. conceived the study and the use of video chat technology to quantify adherence, and C. M. P., J. W. A., C. W. and C. K. M. all contributed to the study design. C. M. P., J. W. A. and C. W. collected the data and were involved in data analysis. C. M. P. and C. K. M. drafted the manuscript, and all authors edited and approved the final manuscript.
The authors declare that there are no conflicts of interest.
Supplementary material
For supplementary material/s referred to in this article, please visit http://dx.doi.org/doi:10.1017/S0007114516003524
