Introduction
We are living in an age of data, where data slowly transcend, creep if you will, into all corners of our lives. As large corporations such as Apple and Google move into the health domain, so do data: the promises of datafication of healthcare around personalized care, improved quality of care, cost reduction, and leaner processes exert pressure on electronic health record management, employee healthcare, and health insurance (Powles and Hodson, Reference Powles and Hodson2017; Sharon, Reference Sharon2021). Increasingly, this development is also blurring the line between clinical and consumer healthcare solutions, particularly related to lifestyle change (Lucivero and Prainsack, Reference Lucivero and Prainsack2015). “P4 medicine” is expected to drastically change the way healthcare is provided and used over the course of the next decade, by making it predictive, preventive, personalized, and participatory (Flores et al., Reference Flores, Glusman, Brogaard, Price and Hood2013; Ruckenstein and Schüll, Reference Ruckenstein and Schüll2016). Similarly, the movement towards Precision Management also aims to bring personalized medicine to the next level through environmental, lifestyle and medical history, and even genomic data in large data sets (National Research Council, 2011; Hogle, Reference Hogle2016). Engaging with large data sets in healthcare inherently means that there is a need to scale up the data infrastructure and data handling processes, and to approach healthcare innovation as the multi-stakeholder, systemic challenge that it is. Various participatory design approaches have been applied in order to tackle healthcare innovation from a more systemic perspective, including strategies such as co-ideation with different stakeholders (Slegers et al., Reference Slegers, Wilkinson and Hendriks2013; Rothmann et al., Reference Rothmann, Danbjørg, Jensen and Clemensen2016), experience-based design (Bowen et al., Reference Bowen, Dearden, Wright, Wolstenholme and Cobb2010), and visual history mapping (Villalba et al., Reference Villalba, Donovan, Jaiprakash, Askew, Roberts, Russell, Crawford and Hayman2018). Another way of engaging with the complexities of systemic healthcare change is through the use of data in intelligent ecosystems. Data-enabled design (DED) (Kollenburg and Bogers, Reference Kollenburg and Bogers2019) has been shown to allow for collecting and analyzing contextual, behavioral, and experimental data that can be connected to medical data gathered in hospitals (Bogers et al., Reference Bogers, Kollenburg, Deckers, Frens and Hummels2018; Kollenburg et al., Reference Kollenburg, Bogers, Rutjes, Deckers, Frens and Hummels2018). Clinicians are already interpreting data sets provided by patients (Rutjes et al., Reference Rutjes, Willemsen and IJsselsteijn2019) and have started using consumer health wearables in their own practice (Jensen et al., Reference Jensen, Treskes, Caiani, Casado-Arroyo, Cowie, Dilaveris, Duncker, Frederix, Groot, Kolh, Kemps, Mamas, McGreavy, Neubeck, Parati, Platonov, Rienzo, Schmidt-Trucksäss, Schuuring, Simova, Svennberg, Verstrael and Lumens2021). However, DED is an inherently malleable practice with plenty of designerly leeway, whereas the clinical context is controlled and protected. This can lead to some friction. In order to truly contribute to the implementation of innovation around using data to personalize healthcare, DED needs to scale up in terms of context, duration, stakeholder involvement, and interactivity.
Aims and strategy
In this article, we discuss the DED process of the ORBIT system, which is an intelligent ecosystem aimed at helping bariatric patients who have had a stomach reduction surgery with lifestyle change to maintain weight loss. This system was designed in collaboration with healthcare professionals at the Catharina Hospital in Eindhoven, The Netherlands. We also describe the proposed setup of a DED evaluative study that was planned with the system but not conducted due to rejection by the medical ethical committee (METC). The study was rejected due to its explorative character, in combination with the collection of a large amount of participant data. The METC raised concerns about the burden of data collection for the participants and had questions about the clinical relevance of the expected results of the study. We address the METC feedback in more detail in the section “METC feedback.”
The clinical context is a highly sensitive design space that can benefit from the application of the DED process. However, as already acknowledged (Kollenburg and Bogers, Reference Kollenburg and Bogers2019), the proceduralized character of the clinical context seems to conflict with the explorative character of DED. Therefore, in this article, we aim to answer the following research question: “How to expand and scale up Data-Enabled Design in order to make it applicable to a clinical context?”
In order to answer this question, we introduce a case study of a structured and well-prepared attempt to bring DED to the clinical context. We hypothesize that DED could be expanded and scaled along four different aspects, which would make the methodology more suitable for the vulnerable context and the unwieldy organization. We intend to keep DED's original intention of being situated and contextual, but make a few suggestions for modifications when applying the methodology in a clinical context. We reflect on how realistic it was to achieve the above goals when designing ORBIT, what they mean for the underlying design process, and present key takeaways around designing with data in the clinical context.
Theoretical background
As this article relies strongly on the DED methodology as coined by van Kollenburg and Bogers in their PhD thesis (Kollenburg and Bogers, Reference Kollenburg and Bogers2019), we use this section to provide some background information on the methodology. Furthermore, we provide some insights on ethical procedures in healthcare and about exploratory data collection in the clinical context specifically. Reading these sections makes it easier to understand the required changes for scaling up and expanding DED into the clinical context.
Data-enabled design
DED describes a situated design practice of using data as a creative material for designing intelligent ecosystems. Intelligent ecosystems are a dynamic composition of interrelated products, services, and people. By collecting data and using artificial intelligence these systems are able to understand and adapt to their users. The authors advocate for a multi-step design approach that consists of a combination of a research-oriented contextual step and a design-oriented informed step. These two steps are combined in a loop (cf. Fig. 1) that tightly links design research and the contextual everyday. An element of automation is also represented in the loop, captured in the dotted line in the upper loop, which stands for gaining an understanding about the context and adapting to it. Eventually, this loop is meant to become self-sustaining, when the solution is embedded in the context and no further involvement from the design research team is required.
The DED loop, taken from Kollenburg and Bogers (Reference Kollenburg and Bogers2019).
DED relies on four main activities to be carried out by a multidisciplinary design research team: (1) rapid and flexible prototyping, where the tested prototypes can be adjusted throughout a design (research) study (step A), (2) data collection and analysis (step B and C), (3) using data as a creative material for designerly explorations and synthesis (step D and E), and (4) (remote) intervention design (step F). The resulting intelligent ecosystems are data-oriented and learn to adapt to their users, initially through a human-in-the-loop, but with the aim of increasing system automation along the way. As the system learns to understand, it provides the researchers with continuous insight into the everyday context of the user (see all steps in Fig. 1).
Research-oriented contextual step
The contextual step is meant to gain an understanding of the everyday context, focused on understanding the values and the motivations of the different people within their context. Usually, the contextual step will focus on collecting research data, that teaches the design research team something about the intricacies of the context, and which provides a baseline for identifying opportunities for design intervention. In this phase, participants will primarily provide data and receive basic feedback on it. One of the outcomes of this step is a data strategy that informs the design research team on what data (not) to collect from the context.
Design-oriented informed step
The informed step follows the contextual step and is more design-focused, introducing interventions into the context. This step can be conducted with the same participants as the contextual step but often a new group of participants is recruited in order to test the assumptions from the contextual step against a fresh pool of participants and their contexts. The informed step is more reactive, not only gathering research data but also what van Kollenburg and Bogers call “solution data” (Kollenburg and Bogers, Reference Kollenburg and Bogers2019). Solution data is data that users of the system (indirectly) interact with, either because it is presented to them for reflection and feedback, or because the data are used to trigger system interactions.
A concrete example: a solution might be a system that suggests recipes. The research data might cover a diverse range of topics such as the weather, the user's calendar, and information about who is at home. Through collecting these data, the team might discover that only the user's calendar influences what kind of recipe they prefer, the solution would then only use that data, significantly decreasing the pool of information used in the final solution. The more extensive research data, collected under controlled conditions, is then essential to limit the amount of information gathered, stored, and used in the final solution.
Ethical procedures
Both designers and clinicians have acknowledged that designing with and for a clinical context is not without friction and that it is in fact a difficult, perhaps wicked, process (Agich, Reference Agich2001; Goodyear-Smith et al., Reference Goodyear-Smith, Jackson and Greenhalgh2015; Taylor et al., Reference Taylor, Crowe, Pujol, Franklin, Feltbower, Norman, Doidge, Gould and Pagel2021). Ethical procedures for clinicians often leave little room for intuition, experience, and knowledge about disease progression and they are often not accustomed to the flexible and unexpected nature of in-the-field studies that are conducted for design research (Munteanu et al., Reference Munteanu, Molyneaux, Moncur, Romero, O'Donnell and Vines2015). Furthermore, they are not accustomed to participatory research that requires collaboration between researchers and their participants (Wilson et al., Reference Wilson, Kenny and Dickson-Swift2017). Before doing a clinical study, it needs to be approved through a thorough medical ethical approval process, which is strict and sometimes unforgiving. Clinicians refer to the Hippocratic oath to “do no harm” to their patients, even if that means that they cannot treat them (Hofmann et al., Reference Hofmann, Burke, Pearlman, Fiedler, Hess, Schull, Hudson and Mankoff2016, Reference Hofmann, Williams, Kaplan, Valencia, Hann, Hudson, Mankoff and Carrington2019). This is in line with scientific research in general, including design researchers who, too, make the promise to “do no harm” when they receive their university degrees (Bruckman, Reference Bruckman, Olson and Kellogg2014).
However, in practice, it is hard to assess the risk of harm being done. Design researchers and clinicians alike have advocated for a change in ethical approval processes, as they see that strict procedures can also create a disincentive to conduct high-quality research” (Bruckman, Reference Bruckman, Olson and Kellogg2014, p. 459). They stress the importance of a dialogue between researchers and the committee (Bruckman, Reference Bruckman, Olson and Kellogg2014; Munteanu et al., Reference Munteanu, Molyneaux, Moncur, Romero, O'Donnell and Vines2015; Wilson et al., Reference Wilson, Kenny and Dickson-Swift2017), as well as a stronger acknowledgement of the flexible nature of participatory design research (Goodyear-Smith et al., Reference Goodyear-Smith, Jackson and Greenhalgh2015; Munteanu et al., Reference Munteanu, Molyneaux, Moncur, Romero, O'Donnell and Vines2015). Therefore, ongoing collaboration with clinicians is required in order to understand results and to find ways for these results to be implemented, which can lead to a bottleneck that slows down the process of understanding and applying design knowledge in the context (Kashfi, Reference Kashfi2010).
It is important to think about the ethical implications of collecting, analyzing, and visualizing personal data and it is of utmost importance to enforce the right to privacy, and the right to the protection of personal data of individuals (Lynskey, Reference Lynskey2015). Based on the GDPR legislation on lawful processing, a legal basis needs to be established for data collection. In order to carry out a clinical trial that involves data collection, among other things, the team needs to carry out two important steps: (1) defining the purpose of data collection and (2) acquiring consent for data collection from the participants (Mondschein and Monda, Reference Mondschein, Monda, Kubben, Dumontier and Dekker2019). It is important to stress that special rules apply for scientific research and clinical trials, because “it is often not possible to fully identify the purpose of personal data processing for scientific research purposes at the time of data collection.”Footnote 1 Nevertheless, the research team needs to follow all recognized ethical standards for scientific research and establish appropriate safeguards for the rights and freedoms of data subjects.
In summary, it is important that we critically review how security, privacy, and ethical compliance procedures are impacting design innovation in the clinical context, and that both sides actively engage in improving these procedures to support innovation in health care.
Data in the clinical context
We are rapidly approaching the datafication of healthcare, where data are not only used in strictly clinical measurements but also to assess personal experiences with health. In current clinical practice, the main way that data are being used to capture personal experiences is through questionnaires. The analysis of these questionnaires [like e.g. SF-36, MA-QoLQII (Mazer et al., Reference Mazer, Azagury and Morton2017), and OBESI-Q (Nienhuijs et al., Reference Nienhuijs, Lindhoud, De Vries, Hoogbergen, Cnossen, Gernette, Akpinar, Montpellier, Van Rossum, Schouten, Pusic, Klassen and Liem2019)] produces a numerical outcome that can be compared before and after clinical care is provided to the patients. However, it is a big challenge to determine whether a (statistically) significant change in the outcome of these questionnaires is clinically relevant. Sloan et al. (Reference Sloan, Symonds, Vargas-Chanes and Fridley2003) have explored the topic in more detail, and while they provide good input on various methods to deal with the issue statistically, they also mention a dilemma that we, as design researchers, can very well relate to: “If we want to find out whether a change is clinically significant, why not just ask the patient? If the patient tells us that a significant change has occurred, it begs the question as to whether we need to apply an abstract number to the concrete concept that some change has occurred” (Sloan et al., Reference Sloan, Symonds, Vargas-Chanes and Fridley2003, p. 28). Villalba et al. (Reference Villalba, Donovan, Jaiprakash, Askew, Roberts, Russell, Crawford and Hayman2018) have also pointed out the need for a stronger focus on “soft” factors when designing with data in the healthcare domain.
When talking about data practices in healthcare, it is becoming increasingly important to not only address technology but also the higher levels of policy and social conditions that make it possible to seamlessly integrate data into everyday hospital practices (Hogle, Reference Hogle2016). Healthcare in the Netherlands is readily available to all citizens at a relatively low cost due to obligatory health insurance and government funding for hospitals (Enthoven and Van de Ven, Reference Enthoven and Van de Ven2007), but this also means that complicated financial and organizational structures are in place. Shifting towards lifestyle improvement and preventive healthcare (Lucivero and Prainsack, Reference Lucivero and Prainsack2015) means that the line between non-patient and patient becomes blurred. Thus, the challenge of integrating data practices into the clinical context is no longer merely a matter of having access to the right technology, it is also about reforming the way that we think about health care, hospitals, and our personal data, and the role that healthcare plays over the course of our lives.
Method
This article provides a reflection on the design process of the ORBIT system, and its proposed explorative, clinical, DED evaluation study by the design research team. For the composition of this team, please refer to Table 1. All activities presented in this article took place over the course of 2020.
Overview of the team involved in the design of ORBIT and the study setup
The clinical PhD would also be involved in the ORBIT system solution in the role of the care coordinator.
The ORBIT system design is a response to earlier work that was done in the Co-Responsibility project (Jansen et al., Reference Jansen, Niemantsverdriet, Burghoorn, Lovei, Neutelings, Deckers and Nienhuijs2020; Pannunzio et al., Reference Pannunzio, Lovei, Neutelings, Deckers, Jansen and Burghoorn2020; Lovei et al., Reference Lovei, Van Dijk, Jansen, Deckers, Niemantsverdriet, Burghoorn, Neutelings and Nienhuijs2020b; Versteegden et al., Reference Versteegden, van Himbeeck, Burghoorn, Lovei, Deckers, Jansen and Nienhuijs2021). The concept of Co-Responsibility has been defined as “responsibilities of people being intertwined, not in the sense that people share the same responsibilities, but in the sense that peoples’ responsibilities are interdependent” (Devisch, Reference Devisch2012; Neutelings et al., Reference Neutelings, Levy, Djajadiningrat and Hummels2017; Jansen et al., Reference Jansen, Niemantsverdriet, Burghoorn, Lovei, Neutelings, Deckers and Nienhuijs2020, p. 1538). Therefore, the partners of the bariatric patients are also important actors of the designed intelligent ecosystem, so that it is possible to assist them in tackling lifestyle change together with their partner.
ORBIT system
ORBIT is an intelligent ecosystem fully embedded in the existing bariatric care pathway of patients. It offers patients (clinically approved) lifestyle support content in the form of personalized care programs to assist them in getting and maintaining a healthy lifestyle after undergoing bariatric surgery (see Fig. 2), to give patients a better outpatient experience, and to help them integrate new habits into their life after their surgery. ORBIT utilizes off-the-shelf data trackers that would be capable of providing the right amount of clinical, contextual, and behavioral data for the ORBIT system to provide the right (clinically approved) content at the right time. By right amount, we mean that the system would gather the bare minimum of data required to assign the appropriate care programs. The ORBIT system is heavily reliant on sensor data, to be collected in the homes of participants. Some of these sensors are open-ended (Kollenburg et al., Reference Kollenburg, Bogers, Rutjes, Deckers, Frens and Hummels2018; Kollenburg and Bogers, Reference Kollenburg and Bogers2019), such that they can be applied in a way that would best help the participants with the unique and personal aspects of their condition (e.g. reporting the level of boredom they feel during a day).
Overview of how the ORBIT system works after the patient's bariatric surgery.
Patients can access the system through an app on their phone, while health care professionals (HCPs) can access the system through an online dashboard. The system content is presented to the patients in different programs that they can follow in the app. These programs offer education and stimulate the patient to complete tasks such as setting personal goals, going for walks, and adapting their sleeping schedule. Multiple sensors are also connected to this system to monitor patient participation in the programs and to provide more tailored feedback on their lifestyle in the moment (e.g. giving a compliment once they reach their amount of steps for the day). The sensors to register this information were carefully selected to be medically approved and clinically reliable, which is essential when using data trackers to draw clinical conclusions (Jensen et al., Reference Jensen, Treskes, Caiani, Casado-Arroyo, Cowie, Dilaveris, Duncker, Frederix, Groot, Kolh, Kemps, Mamas, McGreavy, Neubeck, Parati, Platonov, Rienzo, Schmidt-Trucksäss, Schuuring, Simova, Svennberg, Verstrael and Lumens2021).
The ORBIT system offers a chat functionality, through which patients can directly contact their HCPs. In order to keep this process streamlined, each patient has one dedicated care coordinator who serves as the entry point for the patient. For the purpose of the DED evaluative study, this would be the clinical PhD student who was also actively involved in the content creation for the care programs. The care coordinator manages the medical case of the patient and reviews the data that are being collected about them in their home environment. If the care coordinator would spot any oddities, they could reach out to the patient to ask what is going on, or forward the data to the relevant medical specialist. This would be done by sending care questions from the patients to the specialists through e-mail and (automatically) logging their responses in the platform.
At the basis of the ORBIT system are the patient care profiles, which provide a collection of in-depth information about patient preferences towards treatment, behavior within the different programs, and detailed information about the patient's stage of behavior change, based on the stages of change within the Transtheoretical Model (Prochaska et al., Reference Prochaska, Redding, Evers, Glanz, Rimer and Viswanath2015). Moreover, the ORBIT system stores, organizes, and analyzes data that are collected about the patient in the home environment and at the hospital. The care profile is updated and compared to the patient's baseline and their previous measurements and is then updated in the ORBIT system over time. The care profile elements (e.g. daily number of steps measured by a physical activity tracker) were pre-determined based on co-creation sessions with the multidisciplinary design team creating the ORBIT system, including clinical experts.
Proposed DED evaluative study setup
The proposed study setup was created by a multidisciplinary team with members from the healthcare technology industry, the hospital, and academia. The team had a broad skill set with experience in design research, medicine, data science, and medical ethical procedures – both from an industry and a hospital perspective. We took a DED approach in order to investigate how to motivate and activate the patients and their partners. We had multiple design research goals with this study: (1) measuring Co-Responsibility (as defined above) using the “ORBIT system,” (2) identifying design opportunities for a clinical care pathway, and (3) expanding and scaling up DED for the clinical context. In order to achieve these goals, we intended to introduce a prototype of the described “ORBIT system” in the bariatric care pathway of the Catharina Hospital in Eindhoven (CZE). Six families (12 participants), living in the maximum 25 km radius of Eindhoven were to be recruited. One member of the participating families would have been a bariatric patient who underwent surgery at the CZE 1–1.5 years prior to being included in the proposed study. The partners of these patients would also have been recruited as study participants. The study would last a full year and would consist of four distinct phases: (1) general notion (1–2 weeks), (2) baseline data collection (2 weeks), (3) design-oriented informed step (2–2.5 months), (4) follow-up measurements (1 year after study start). As it can be seen the duration of going through the DED loop (cf. Fig. 1) was supposed to take 3 months (phase 2 and 3). The proposed study was classified as a clinical trial. Under the Dutch law, any clinical trial in the Netherlands needs to be approved by an METC before recruitment and execution. The committee is composed of a group of independent specialists, and researchers can choose to apply to one of the nationally approved committees.
METC feedback
The proposed study setup was eventually rejected by the METC. In our case, the METC rejected the study based on doubts regarding the clinical relevance of the proposed method and results, as well as a concern about the burden of participation in the research for recruited participants. The method was criticized as the quantitative measurements that we proposed did not fit the qualitative and narrow research question, where an explorative study was considered more suitable. The committee also did not find it credible that the research question could be answered with the suggested measurements, which led them to question how the study would make a meaningful contribution to the medical field. Similarly, it was confusing to them that the study contained elements of exploration, implementation, as well as evaluation.
The burden for participants was also considered too high, due to the amount of data trackers, as well as the privacy-sensitive information that would be collected by the trackers and through the interview questions. Additionally, the METC did not find that the benefits that participants would receive weighed up against the burden. They judged that the same information could be gathered in far less intrusive ways, for example through questionnaires.
We do not intend to criticize the METC decision in this article. In fact, we agree with the nature of the concerns that the committee posed. Instead, we want to reflect on the result as there are many valuable lessons to be learned from the study design and the process towards the METC submission and eventual rejection. Despite the METC feedback, our clinical partner remained positive about our collaboration and approach.
Contribution
This paper was written as a reflection by (part of) the design research team involved in the design of the system and the evaluative study with it. Over the course of one year, we completed a large number of design activities, including (remote) workshops with HCPs from the departments we were working with, writing the research protocol, and prototyping and evaluating the system. This system included the app and sensors to be used by patients, the dashboard and e-mail structure to be used by the HCPs, and the infrastructure to support it all. We analyzed how we applied the DED process, and which changes were made in order to use the approach in the clinical context. This resulted in four considerations with regards to using DED in a clinical context, that we propose as ways to move the methodology forward into the clinical context.
Results
In this section, we answer our research question about the ways to expand and scale up DED in order to apply this design process in a clinical context by introducing four recommendations to modify DED. The recommendations are presented in the following structure: we first look at how we intended to apply “Data-enabled design” in the proposed evaluative study of the ORBIT system. Secondly, we reflect on our learnings and related work to identify what the current boundaries of DED are and what boundaries we faced. Finally, we conclude with our recommendations for conducting a better DED process in the clinical context.
Our first modification is that of lowering the barrier of entry required to start a design-oriented informed step (see Section “Design-oriented informed step”). Secondly, we argue that an increased level of system automation in the designed intelligent ecosystem can be introduced. Thirdly, we aim for increased involvement of clinical stakeholders in the DED process. Finally, we show that DED could benefit from a more detailed account of the short-term and long-term future. Reflecting on these four ways of scaling up and expanding the DED process, we compare the execution of the ORBIT design process to the original DED loop (see Fig. 1) to then draw conclusions about how to move DED forward in a clinical context (Fig. 3).
The four extensions, sketched out in relation to the original DED loop as presented in Figure 1.
Lowering the barrier to enter a design-oriented informed step
One of the key outputs of the research-oriented contextual step (see Section “Research-oriented contextual step”) is an overview of data that is essential to collect. Moreover, it helps the design team to make a distinction between data points that are necessary for the designed solution and data points that are collected for (design) research. After the contextual step, the design team is able to build a data strategy for the informed step and via the design synthesis step create or mature the means for collecting these data. In the clinical context specifically, a lot of contextual work can also be taken from existing medical research that is widely available in medical journals.
Building on previous research (Jansen et al., Reference Jansen, Niemantsverdriet, Burghoorn, Lovei, Neutelings, Deckers and Nienhuijs2020; Pannunzio et al., Reference Pannunzio, Lovei, Neutelings, Deckers, Jansen and Burghoorn2020; Lovei et al., Reference Lovei, Van Dijk, Jansen, Deckers, Niemantsverdriet, Burghoorn, Neutelings and Nienhuijs2020b), we decided to focus on the informed step (see Section “Design-oriented informed step”), instead of repeating the research-oriented contextual step. This meant that rather than investigating the context again first, we took the insights gained from the previous study as the baseline for our design work, essentially starting at the bottom of the 8-loop instead of at the top. Instead of starting with prototypes situated in everyday life (step A), we went straight for the design synthesis step of the new study (step D), using the insights into context, behavior, and experience of past participants (see Fig. 4). This decision was made as the clinical partner, the patient group, and the bariatric care pathway were mostly the same for both studies. Therefore, our hypothesis was that the collected insights during the previous Co-Responsibility study could be reused and would speed up our design synthesis step. The deviations from the previous research were supported by scientific publications, which were often presented to us by our clinical partners.
The loop projected onto the stages of the ORBIT design process, with the Co-Responsibility project represented on the left and the ORBIT process on the right.
Approaching the boundaries of DED
The Co-Responsibility study (Jansen et al., Reference Jansen, Niemantsverdriet, Burghoorn, Lovei, Neutelings, Deckers and Nienhuijs2020; Pannunzio et al., Reference Pannunzio, Lovei, Neutelings, Deckers, Jansen and Burghoorn2020; Lovei et al., Reference Lovei, Van Dijk, Jansen, Deckers, Niemantsverdriet, Burghoorn, Neutelings and Nienhuijs2020b) shows which data types are relevant and how to collect it in the context of post-operative care after bariatric surgery. ORBIT was designed using this data strategy and applying it for a more complex clinical setting. We soon discovered that while the HCPs were willing to move on with the system development, they were hesitant with regards to the burden and purpose of data collection, as was the METC. This was an unexpected result to us, as in the previous study more data had been collected than what we were asking for now. In fact, in the Co-Responsibility study, the participating patients were asked about whether they would agree to share their data beyond the research setup. They said they would do so for a trusted party that can interpret their data (e.g. a case manager at the hospital). So this was what we took as a design consideration when proposing our DED evaluative study. This step towards implementation made all of us think differently about its consequences, triggering new types of reflections and insights. Discussions about patient privacy, burden of data collection, and practicalities around the implementation of the system took the foreground, instead of the inquisitive and curious attitude that dominated in the contextual step. The result of the project was that we needed to go back to the drawing board and perform some extra contextual activities to iron out our expectations of the system. We ran “deep dive sessions” with multidisciplinary design experts, data scientists, and clinical experts. This is how we learned that having a solid data structure is not the only requirement for moving into the informed step, but that it requires conceptual preparation and preparation in the existing culture as well. As we moved from gathering research data for research purposes into gathering solution data that would become an inherent part of healthcare practice in the future, a tension arose. The evaluation of our methods by the HCPs and the METC led to demands that counter the intentions of DED as an open-ended methodology, as we seemed to be proposing a solution. DED depends strongly on quick iteration and adaptation, and even while performing an informed step, the designed intervention still holds many uncertainties. In order to be able to move through a full 8-loop, it is important that all stakeholders are on the same page about the stage that the project is in.
Recommendations
To lower the effort of executing a design-oriented informed step, we decided to use the output of a previous study's contextual step. An intelligent ecosystem and a data strategy existed from the previous study. We expected this to speed up the process towards the informed step, where the system is not only informed by the contextual step, but also by an understanding of the existing data structures and organizational constructs. Among the team members, implementation discussions dominated and overshadowed the focus on the design work that was also required. In order to avoid this risk, we suggest for other design teams that would like to follow our steps to be mindful and critical about their data strategy and to consider what it means to move from collecting research data to solution data. More extensive data collection in a contextual step might be acceptable, in line with the GDPR principle on data collection for scientific purposes. However, once a project moves into the informed step and the data become more solution driven, the ethical considerations also need to shift from research-related ethical considerations to market-related ethical considerations, where the study system should be approached as if it is the final solution. When serious ethical implications (e.g. burden of data collection) arise during the setup of an informed step, the team has to consider increasing the amount of “design plumbing” (Lovei et al., Reference Lovei, Deckers, Funk and Wensveen2020a), multi-stakeholder co-creation, and other means to decrease the burden and sharpen their data strategy before moving towards a successful design-oriented informed step.
Increasing system automation
In DED, we are collecting behavioral, contextual, experimental, and clinical data. We combine, analyze, and understand the multiple data sources. We are using these data collected from the real-life context in order to start building the necessary intelligence elements. We create prototypes that use these data even if the algorithms and intelligence that bring us to the insights necessary to deliver design interventions that do not exist yet. Nevertheless, many of the underlying processes can be automated to facilitate a smoother and scaled up DED process. Therefore, we suggest to increase the amount of system automation that is present in the DED process in order to enter the clinical context. In this section, we discuss the risks, benefits, and opportunities of automation in DED.
When we look back at the DED 8-loop (Fig. 1), inside the upper loop there is an intelligent ecosystem we are designing. Implementing and improving the automated elements of this system can be seen as an effort to shrink the lower loop of the system until it disappears completely. We visualized this through the arrows in Figure 5.
The envisioned automation within the DED process, where the design (research) loop shrinks due to the continuous effort of automation, especially in the backend.
From an automation standpoint, the clinical context poses a challenge as a complex infrastructure is already in place, which cannot be easily changed for the purpose of a DED evaluative study. Based on human-centered artificial intelligence principles (Shneiderman, Reference Shneiderman2020) we aim at designing an automated intelligent ecosystem that enables a high level of human control and a high level of automation. In order to facilitate this high level of human control, the proposed ORBIT system includes the system used in the hospital for electronic patient records, a patient portal that patients use to get insights into their own medical information, and also the highly protected e-mail clients and computers at the hospital which were needed to communicate with stakeholders and potential participants (patients) in the DED evaluative study.
Approaching the boundaries of DED
With the ORBIT system, we set out to increase system automation, as a way to make it easier to scale up the system to future inclusion of more participants, as well as reduce the burden on our clinical partners regarding system and data management throughout the study. In previous DED studies (Bogers et al., Reference Bogers, Kollenburg, Deckers, Frens and Hummels2018; Kollenburg et al., Reference Kollenburg, Bogers, Rutjes, Deckers, Frens and Hummels2018; Jansen et al., Reference Jansen, Niemantsverdriet, Burghoorn, Lovei, Neutelings, Deckers and Nienhuijs2020), much of the system management was done manually, making it easier to adapt system behavior along the way and to respond to unexpected situations. The proposed DED evaluative study with the ORBIT system focused on the design-oriented informed step. Therefore, we hypothesized that the system would need to be stable and that the infrastructure should be integrated with the hospital's existing infrastructure to be effective. To this extent, we involved the clinicians in our brainstorming sessions and designed parts of the care path with them.
To establish a connection between the hospital's existing system infrastructure, we uncovered a couple of systemic challenges. The high level of security sometimes makes it impossible to open external links on the computers in the hospital, and the security restrictions on patient data prevent some data from being sent outside of the hospital, which meant that it could not be directly shared with us. When performing a clinical study, this means that a prototyped system can never exist on its own and that data will always have to be transferred between the new, prototyped system, and the existing system: legacy systems continue to be used and relied on across the hospital as they hold all information relevant to patient treatment, especially for those departments that are not involved in the clinical study.
Recommendations
With every full loop (see Fig. 1) the aim is to eventually shrink the design (research) loop and to push everything towards the everyday life loop, such that the designed intelligent ecosystem becomes self-sustaining. Automation is an active effort towards shrinking that loop, step by step. However, excessive automation in the clinical context can be really dangerous. Unfortunately, we saw in the case of the ORBIT system that it is not yet possible to introduce an increased level of automation into the current clinical infrastructure without safely giving up control. It is, however, possible to design around it and use the collected insights for making a plan on how to further automate the work of the healthcare professionals in a safe and secure way. It is advisable when applying the DED process in a clinical context to make a clear plan on what can be automated and what not, and also examine in detail the feasibility of the introduction of such automation in its intended context by taking into consideration which systems are already in place.
Scaling up the clinical involvement
DED is essentially a multi-stakeholder design process, where very different types of stakeholders are involved in the same system, either as users, as data analysts, or as designers. By including experts and end users in the design process of an intelligent ecosystem, it becomes possible to create a dialogue and take into consideration the different perspectives to be addressed in the designed solution.
In the original DED 8-loop, there is a very clear distinction between the upper and the lower loop, between the context and the design research studio. This applies to many contexts, especially those where designers are directly designing for their end users. However, in a clinical context, this is slightly more complex, as the involved clinicians can take on roles on both sides of the loop. This requires careful selection of the parties involved and a clear agreement on the type of role that each stakeholder plays.
With setting up the DED evaluative study, we were extremely lucky to be enthusiastically supported by two leading physicians in the hospital who strongly believed in our methodologies. Their trust was built with the previous study (Jansen et al., Reference Jansen, Niemantsverdriet, Burghoorn, Lovei, Neutelings, Deckers and Nienhuijs2020; Pannunzio et al., Reference Pannunzio, Lovei, Neutelings, Deckers, Jansen and Burghoorn2020; Lovei et al., Reference Lovei, Van Dijk, Jansen, Deckers, Niemantsverdriet, Burghoorn, Neutelings and Nienhuijs2020b; Versteegden et al., Reference Versteegden, van Himbeeck, Burghoorn, Lovei, Deckers, Jansen and Nienhuijs2021), as well as through their devotion to healthcare innovation. Their dedication meant that they were involved throughout the process, and got many of their colleagues involved as well, including nurses, nurse practitioners, and clinical PhD students. Besides their willingness to innovate health care, they too were driven by research goals and a desire to publish in medical journals about this new way of working.
Approaching the boundaries of DED
Throughout the design process of the ORBIT system, there were multiple occasions on which we involved the clinicians in the process. These included events such as full-day workshops, brainstorming sessions, and in-depth sessions around the system content. Eventually, we realized that we were already going through the DED loop, as we were going back and forth between the design research studio and the context. The gathered data might not have been sensor data yet, but a lot of information was gathered from the HCPs in each interaction. However, we also realized that this meant that we were in fact introducing a third loop into the DED 8-loop (as portrayed on the left side of Fig. 6). As the clinicians we were collaborating with were also an essential part of the end solution, a lot of our interaction with patients had to go through the collaborating hospital for validation, either through the involved clinicians or through the METC. In some cases, if the content had been validated before the deployment of the system, we might be able to bypass clinical control occasionally, as represented by the arrow on the right. However, here we also acknowledge that for the system to work, it had to be done through the HCPs, and that in order to prepare for a more permanent solution, they needed to have an important role in the study procedures as well.
The clinical involvement is represented within the DED loop. Left: as the ORBIT system was designed, where the clinical context forms a validating factor between design researchers and context. Right: the ideal, envisioned situation, where clinicians are an active part of both worlds.
Eventually, one of the steps we took to overcome this hurdle more easily was to ask the clinical PhD student to take on an active role within our design research team. When she took on this role and joined us for creative sessions in our office, we embedded part of the medical domain into our design research studio. Doing so significantly increased the clinical relevance of our work, as well as the speed at which we could make new steps. Through her presence, we not only had access to her medical knowledge but also to that of her colleagues, whom she contacted whenever relevant questions arose. Therefore, we learned to not only involve clinicians as part of the design context but also as an invaluable part of the design team.
Recommendations
Reflecting on the left loop in Figure 6, we also considered what the ideal collaboration with the clinicians would be, as represented in the right drawing. The goal of the clinical involvement is to create a DED loop specifically related to the content inside of the existing big loop, where HCPs can be part of the design team, as well as the solution. This smaller loop also represents the assumption that several iterations of the loop with the clinicians are needed, next to the loop where patients are involved. Having this close connection, and multiple explorations of the system with the clinical staff are important elements of the DED process within the clinical context, as we made significant steps on the system design, even when we did not yet get round to interacting with patients. The clinical context is, therefore, harder to fit into the original DED loop, as instead of two stakeholders (the design researchers and the participants), it now involves three: the design researchers, the patients, and the HCPs.
Engaging with the future
DED allows study participants to experience a new reality, by introducing new components into the participant's ecosystem. Through these interventions, the design research team and the participants are able to shape this new reality together. The existing DED framework is deliberately unspecific about promises and future prospects of interventions, as the need for interventions emerges directly from the context. However, for a complex technological, social, and organizational transition such as the innovation of care at large, we cannot shape the innovation with data collected from the patient context alone. We need more abstract data regarding the organizations involved and their plans for the future as well, such as the organizational data that Villalba and colleagues used in their work on data timelines (Villalba et al., Reference Villalba, Donovan, Jaiprakash, Askew, Roberts, Russell, Crawford and Hayman2018). Ultimately, designing for larger, organizational issues requires a sharp vision.
To shape this vision we need to engage with the future on multiple levels (Fig. 7), for which we need slightly different tools than DED conventionally uses. Prototyping in the clinical context results in a situation that is much further away from the intended situation than is ideal for validation, due to protocols and procedures that simply cannot be changed for the sake of an explorative study. If we want to get past that, we need to define a course of action over a longer time span and innovate not only on a technological but also on a social and organizational level. This can be done through methodologies, such as data timelines (Villalba et al., Reference Villalba, Donovan, Jaiprakash, Askew, Roberts, Russell, Crawford and Hayman2018), design futuring (Kozubaev et al., Reference Kozubaev, Elsden, Howell, Søndergaard, Merrill, Schulte and Wong2020; Smith and Ashby, Reference Smith and Ashby2020), or speculative design (Dunne and Raby, Reference Dunne and Raby2013) to “determine what citizens and patients desire from their doctors, nurses, services and technologies” (Strachan, Reference Strachan2016, p. e17).
Visualization of how different versions of the DED loop can exist next to one another (time progressing from left to right), where instances are clearly split up, and envisioning future loops can inform current loops.
Approaching the boundaries of DED
For the ORBIT system design, we needed a vision on the future of healthcare shared by all stakeholders to breach the gap from research data to solution data. This vision included decisions about who would be eligible to use the ORBIT system (all patients, a selection of patients, anyone who is concerned about their health), how the system would eventually be financed (the hospital, the patient, insurance), and the level at which the system would be automated (administration, diagnosis, treatment). During the first months of the project, we organized several activities aimed at scoping the future vision of the ORBIT system. Much of the initial framing was done when writing the research protocol for the METC submission, which was largely focused on the short-term implementation of the proposed system. Later, we engaged in more creative sessions which included development sessions with the technology team, content workshops with the clinical team, and “deep dive sessions” with the core team. Especially the sessions with the clinicians helped us to connect to the context and to gain a deeper understanding of upcoming clinical innovations and trends in care.
However, as the study preparations were running at the same time, conversations about the ORBIT system and its implementation in healthcare often became practical, addressing difficulties with implementing the suggested data structure in the hospital's existing systems in the short term. While these were important questions, they were sometimes distracted from the bigger picture. The practical nature of DED means that it requires a lot of planning and preparation, hence there is less time for these discussions. This is why we suggest to expand the DED process, such that futuring and discussions about the envisioned system become an inherent part of the design process.
Recommendations
One of the main recommendations based on the ORBIT design process is that it is extremely important to make a distinction between the near future and the long-term future and to find the time to design for both. By defining a timeline for the bigger transition and deliberately splitting it into different horizons, it becomes easier to talk about the individual horizons and to find relationships between them. Additionally, formulating and shaping the future vision with help of storytelling and scenario building can help validate ideas with future users, both HCPs and (future) patients. In a field where real-life validity is hard to reach due to existing infrastructures and protocols, experiential storytelling could capture a reflection of what real-world implementation might look like. Furthermore, visions, and a concrete representation of them in a narrative, could help health care organizations to see the long-term consequences of short-term innovations and open up discussions at a higher level, towards hospital boards, insurers, and the government. These methodologies could be considered during the design synthesis step in the DED process, as an extension, or clarification, of the applicable methods to process data from the context. Especially issues that are outside of our direct control, such as organizational and financial structures, can then be explored at a deeper level, to understand steps towards further innovation and implementation.
Discussion
While we would have liked to see a different outcome for the METC assessment of the DED evaluative study, we still consider the proposed study to have brought us closer to the successful implementation of DED in the clinical context. In this article, we present an open reflection on the extensions we made to DED in order to learn from the different elements of the process.
Throughout the previous sections we have provided different focused recommendations for applying DED in a clinical context: first of all, we proposed that having a data strategy based on an earlier research-oriented contextual step can speed up the process towards the design-oriented informed step. However, it is important to tread carefully because the existing data structures and organizational constructs can end up requiring additional contextual steps before starting an informed study in the clinical context. As such, it would be helpful to construct a comprehensive overview of the care pathways that are the subject of the intervention. These would address the context at a higher level to not only gather contextual information from the users of the system but contextual information of the system and the organization itself as well. This brings us to our second point, which is about automating the system to make it ready for implementation. We consider that with every full DED loop, the aim is to shrink the lower DED loop more and to push the system towards the upper loop, such that it becomes eventually self-sustaining. Automation is an active effort towards shrinking that loop, step by step. For this recommendation, it will be important to have a stable, yet versatile infrastructure that is supported within the clinical context. More attention is required for the informed step and implementing interventions, which is supported by freeing up resources around the contextual step. Thirdly, we formulate the goal for the clinical involvement to create a DED loop specifically related to the content inside of the existing big loop, where HCPs can be part of the design team as well as the solution. Research protocols should be written in collaboration, and design decisions should be made together with the clinical team. Additionally, the role of the HCPs on the side of the context needs to be considered as well, where they are approached as part of the solution, and therefore also considered users of the system, where their staff experience needs to be taken into account in the design of the system too. This includes their workload, the way in which new technology changes how they execute their work, and possible changes to the organizational structure. Lastly, we consider the importance of defining a timeline for the bigger transition, and deliberately splitting it into a series of future horizons, such that it becomes easier to talk about the individual horizon and to relate it to the other horizons. This helps contain the systemic implications and mitigate perceived and actual risks in DED activities that are projected onto specific horizons. Thinking ahead of the current developments is essential to achieve medical innovation as the approval processes take up significant amounts of time. However, it is even more important to make sure that future visions are scoped and defined in units that are manageable and feasible for direct implementation. One way to do this is by creating narratives for the envisioned future situation and then mapping this onto a timeline, with milestones along the way, specifically placed in the near future.
Overall, we argue that DED, as a highly flexible and context-oriented design method, fits the general trend of datafication in healthcare. Despite the advantages of using data in design and basing design decisions on solid contextual evidence, we also demarcate the likely boundaries of the method that are particularly clearly drawn in the medical clinical context. The METC rejection indicates that we were touching the boundaries of what the DED methodology can do and reach at this stage, which allowed us to deconstruct the main premises of DED – breaking it up – and then investigating how a better fit with the domain can be synthesized in a participatory way. Our recommendations are in part actionable, in part point to further design research necessary for viable and convincing methodological propositions. Ultimately, the rejection is a set-back for the study instance, but at the same time a strong motivation to explore what DED and its tools hold for designing and innovating in complex, constrained, wicked, and highly regulated contexts such as the clinical context.
Even though we view the scope of this paper to be limited to the design process preceding the study with ORBIT, we cannot deny that the rejection of the study has an impact on our observations. Despite the value that we see in our observations, and the value that we are seeing with regards to our contribution in other ongoing projects, we have not (yet) shown these methods successful in practice. This is why much of our future work will be dedicated to further refining and testing out these recommendations in practice – and with that expand the boundaries of DED.
Conclusion
In this article, we have reflected on the DED evaluative study with the ORBIT system as an example of our attempt of breaking up DED and this way exploring how it could enter the clinical context. We have presented findings related to our own experience, and have visualized those according to the original DED loop. As a result, we presented four recommendations to be taken into consideration when applying DED in a clinical context, related to gathering contextual information, increasing system automation, involving clinicians in the design process, and envisioning the future. Learning from the study setup and the journey of the METC approval process, our future work will continue in similar directions and lines of thought: we seek to expand the DED framework to make it suitable for use in clinical settings with the participatory involvement of stakeholders, especially clinical domain experts. By following and evaluating our own recommendations in this article, we aim to show DED as a very viable clinical innovation method for the future.
Acknowledgements
We wish to thank all of our colleagues at Philips Experience Design who played a role in setting up the ORBIT study, specifically Jos-marien Jansen, Anne Wil Burghoorn and Ruben van Dijk. Despite the study not taking place, we have together learned valuable lessons in this project that will help (and are already helping) us to further implement data-enabled design in the clinical domain. This would not have been possible without the time and energy that so many of our colleagues also invested. We also want to explicitly thank the clinicians we worked with for making the time to work with us on the project, in particular Yentl Lodewijks and Simon Nienhuijs.
Competing interests
The authors declare no competing interests.
Renee Noortman is a doctoral candidate at Eindhoven University of Technology in the department of Industrial Design. She researches the role of qualitative data and storytelling in data-enabled design processes.
Peter Lovei is a doctoral candidate at Philips Experience Design and at Eindhoven University of Technology in the department of Industrial Design. He works on scaling up data-enabled design with a focus on micro intelligences.
Mathias Funk is Associate Professor in the Future Everyday group of the department of Industrial Design at Eindhoven University of Technology. He leads the Things Ecology lab and is interested in design theory and processes for systems, designing systems for musical expression, and designing with data.
Eva Deckers is Design Director of the Data-Enabled Design team at Philips Experience Design, where she focuses on shaping transformation in the health technology domain, and on delivering meaningful solutions that encompass data and AI.
Stephan Wensveen is full professor of Constructive Design Research in Smart Products, Services and Systems at the department of Industrial Design at Eindhoven University of Technology. His interest is in using the power of research through design to foster collaboration between research, education and innovation.
Berry Eggen is Full Professor of Design for User Experience in Ambient Intelligent Systems. In addition, he is adjunct professor at the School of Software, FEIT, at the University of Technology Sydney, Australia. Eggen has an interest in ubiquitous computing, multimodal interaction (including intelligent lighting and sound design) and seamless interaction design for everyday life.
Open access
