Impact statement
Decision support system tools are often delivered by research projects for translating scientific knowledge into practical applications, but they are rarely used outside or after the project. Limited reuse may derive from tools that incorporate case-specific data or that are not adequately documented, or that become obsolete in their application context or codes. The case we examine here is that of MULINO-DSS (mDSS), which was designed and released by a European project that ended more than 20 years ago and is still in use. It has thrived for over two decades for at least two reasons: first, it still provides useful functionality to support environmental and integrated decision-making, and second, its authors have maintained and renewed the code through a long series of project grants. The main strengths of mDSS can be found in its reference to a simplified conceptual framework that facilitates communication with interested parties, its flexibility and ability to approach a wide variety of decisional issues, and its relatively simple and understandable decision rules grounded in multi-criteria analysis methods. However, these strengths come with the cost of the effort and skills needed to tailor the system to new applications, in particular for interfacing the DSS with external models and data.
Introduction and background
The water policy background
Water resources management (WRM) is a topic that captures the interests of both policy-makers and the public. Water is crucial for sustaining of life and has significant environmental, economic, and cultural implications. Nevertheless, the outcomes of the 2023 United Nations Water Conference (UN, 2023) showed that the world is not on track to achieve water-related Sustainable Development Goals and targets (UN, 2015), and the global water crisis is worsening.
Over the past few decades, competition for and conflicts over water have arisen as a consequence of increasing anthropogenic pressures (demographic growth, agricultural and industrial exploitation, pollution, etc.) and environmental change. Thus, the need has emerged for new paradigms and methodological frameworks that can support accurate and comprehensive policy instruments and management practices. One of these is the integrated water resources management approach (IWRM), which, according to the Global Water Partnership (GWP, 2000), is “a process which promotes the coordinated development and management of water, land and related resources, in order to maximise the resultant economic and social welfare in an equitable manner without compromising the sustainability of vital ecosystems”, assisting countries “in their endeavour to deal with water issues in a cost-effective and sustainable way” (GWP, 2000).
Based on IWRM principles, in late 2000 the European Union issued the Water Framework Directive (WFD) (EC, 2000), with the aim to achieve sustainable management and the improvement and protection of the quality of European water resources. It operates though six-year cycles, calling for river basin management plans to be developed by river basin districts, adopting an integrated approach that draws on various disciplines including geography, ecology, economics, and sociology. The following are some of the main principles of this legislation:
-
• The geographic management unit should be the river basin district, and planning activities should be articulated at this level (Articles 3, 5, and 13).
-
• The management objectives should include the achievement of good ecological status; therefore, they should take into account a systemic approach to planning in accordance with the environmental standards set out in the directive (Article 4).
-
• The recovery of costs should be one of the objectives that guide pricing mechanisms (Article 9).
-
• The participation of the public in the establishment and updating of river basin management plans should be supported by the provision of information about the planned measures (Article 14).
In 2019, a regulatory fitness check recognised the WFD as a global model for water governance (EC, 2019); it acknowledged progress in water quality prioritisation, improved transboundary cooperation, and enhanced international networks. In the same year, the European Commission took a new step towards sustainable development with the launch of the European Green Deal (EGD) to tackle climate- and environment-related strategies, with water resources at the core. The EGD seeks to transform the European Union (EU) into a fair and prosperous society with a modern, resource-efficient and competitive economy that has no net emissions of greenhouse gases in 2050 and where economic growth is decoupled from resource use. The EGD also reframes sectoral policies and accelerates the implementation of the EU’s water laws. It does so through stronger integration with the Common Agricultural Policy and the Farm to Fork StrategyFootnote 1 (a new comprehensive approach to how Europeans value food sustainability); the Biodiversity Strategy for 2030,Footnote 2 which calls for significantly greater emphasis on nature protection and restoration, including of aquatic and marine environments; and the new Circular Economy Action Plan,Footnote 3 which focuses on a number of water-relevant sectors (textiles, food, plastics, etc.) and promotes water reuse and efficiency.
Decision support tools for sustainable water management
It is evident from the evolution of EU and international policies such as the United Nations’ Agenda 2030 (UN, 2015) that integrating with and contributing to sustainable development are key policy requirements and ones for which water resources are always crucial. However, the paradigms so far adopted, including the IWRM, are debated by many scholars, some of whom question their effectiveness and feasibility in the real world (Biswas, Reference Biswas2008; Garcia, Reference Garcia2008; McDonnell, Reference McDonnell2008). Concerning the IWRM, it is evident that its successful implementation in a specific region requires the integration of multiple disciplines, including hydrology, ecology, economics, and social sciences. In addition, expertise in institutional and legal matters is necessary, and the preference for bottom-up participative approaches over top-down governmental ones should be emphasised (Wilson, Reference Wilson2004, Pahl-Wostl and Borowski, Reference Pahl-Wostl and Borowski2007). Therefore, it is also clear that achieving these goals implies substantial theoretical and practical challenges, requiring comprehensive methodological frameworks as well as capabilities to manage policy- and decision-making processes needing substantial resources, skills, and time. This is the case with sustainable water resources management in complex social and ecological systems, which requires extensive knowledge that encompasses both qualitative and quantitative aspects, such as stakeholder views and perspectives, time series, and spatial socioeconomic and climatic data.
Decision support systems (DSSs) have developed over the past 50 years to address these challenges (Gorry and Scott-Morton, Reference Gorry and Scott-Morton1971). They have been especially useful in unstructured or even “wicked” (Rittel and Webber, Reference Rittel and Webber1973) decision-making processes, as they offer a comprehensive package that includes analytical techniques and models, data management functions, decision rules, and – notably – a user-friendly interface (Keenan, Reference Keenan1998; McIntosh et al., Reference McIntosh, Giupponi, Voinov, Smith, Matthews, Monticino, Kolkman, Crossman, Ittersum, Haase, Haasek, Mysiak, Groot, Siebern, Verweij, Quinn, Waeger, Gaber, Hepting, Scholten, Sulisu, Delden, Gaddis, Assaf, Jakeman, Voinov, Rizzoli and Chen2009; Candido et al., Reference Candido, Coêlho, De Moraes and Florêncio2022). This combination empowers users to make informed decisions efficiently. In the water domain, a DSS can be defined as a computer-based tool consisting of a combination of three main components: (i) simulation models, (ii) user interfaces, and (iii) techniques for decision analysis (Giupponi, Reference Giupponi2014).
Modelling functions provide the simulation capabilities that are needed to explore and anticipate the expected consequences of choices made under the influences of exogenous drivers such as climate and macroeconomic trends before they are made. Decision analysis techniques are employed to facilitate transparent and unbiased judgements and to aid in making informed and rational choices, particularly in situations involving trade-offs and conflicting interests. The most common decision context is the identification of a preferable solution within a set of plausible ones. For example, in the context of the WFD, DSS can be used to identify the most cost-effective combination of measures to meet the objective of good ecological status of water bodies. Multi-criteria analysis methods (MCAM) (Figueira et al., Reference Figueira, Mousseau and Roy2005) are used to consolidate the multiple dimensions of problems and offer a comprehensive synthesis (Giupponi, Reference Giupponi2014). In particular, MCAM offers a wide array of techniques for eliciting and aggregating decision preferences, making it well-suited for cases where multiple incommensurable and often conflicting dimensions, objectives, and interests must be considered (Perosa et al., Reference Perosa, Seitz, Zingraff-Hamed and Disse2022). As a final result of the decision support process, one or more options emerge as those preferred by the community of actors involved. The outcomes of decision support should be adequately documented, and assumptions, subjective choices, and uncertainties of various kinds should be effectively communicated with reports, charts, tables, and/or statistical annexes.
Over the decades, climate change has emerged as a major concern for human development in general and water management in particular. As a consequence, water DSS tools have been recently combined with, or evolved into climate services to improve the quality of information and the analytical capabilities regarding historic climate records, extreme events, forecasts, projections, and – importantly – scenario analysis (Street et al., Reference Street, Buontempo, Mysiak, Karali, Pulquério, Murray and Swart2019, Reference Street, Alterio, Hewitt, Golding, New and Mysiak2022).
Related to scenario analysis and, more broadly, to our limited capability of foreseeing and understanding the future is the growing issue related to uncertainty analysis. Uncertainty and the sensitivity of assessments and decisions to sources of errors and unknowns are crucial issues that permeate all aspects of environmental policy and decision-making. Methods have been developed to address these challenges, such as the robust decision-making (RDM) approach (Lempert and Kalra, Reference Lempert and Kalra2013) and in some cases they have been implemented in DSS tools, to explore how decision options could perform across a multitude of possible scenarios and whether they can reach the required goals in the face of an uncertain future.
Searching the Scopus bibliographic database for “decision support system” retrieves over 150,000 articles from a wide range of scientific disciplines, with computer sciences, medicine, and engineering being the most prominent fields. Environmental sciences represent a relatively small fraction, accounting for only about 5%. When combining the search with “water resource management”, the number of articles is reduced to around 2,600 (since the 1980s). The number of articles experienced growth during the 1990s and peaked in the mid-2000s. Since then, it has remained relatively constant, with an average of around 120 articles per year. This illustrates a sustained interest in DSS technology while adapting to specific focuses and incorporating new simulation techniques and technologies.
However, notwithstanding the evident potential of DSS tools, their role and effectiveness are still debated. Over the years, the literature on DSS has evolved significantly. Initially, it focused on providing science-based methods and data management capabilities to identify optimal solutions for specific problems. However, the emphasis has shifted towards enhancing participatory decision-making processes, with an increased focus on knowledge sharing and managing conflicts and trade-offs (Guimarães Pereira and Corral Quintana, Reference Guimarães Pereira and Corral Quintana2002). Some authors have even considered DSS as learning tools, still potentially quite useful but not directly affecting the decision-making process (Walker, Reference Walker2002; de Kok et al., Reference de Kok, Kofalk, Berlekamp, Hahn and Wind2009).
Some recent reviews have delved into the adoption and performance of existing DSS in subdomains of water resource management. Mabhaudhi et al. (Reference Mabhaudhi, Dirwai, Taguta, Sikka and Lautze2023) examined the use of DSS in agricultural water productivity and showed that less than half of the reviewed DSS tools were accessible in the public domain, which constrained their uptake and the possibility of conducting systematic reviews. Dąbrowska et al. (Reference Dąbrowska, Menéndez Orellana, Kilian, Moryl, Cielecka, Michałowska, Policht-Latawiec, Michalski, Bednarek and Włóka2023) instead focused how cities address water scarcity and flooding issues and found that DSS tend to concentrate on specific problems, primarily water supply, and often do not take a holistic approach to the urban ecosystem. Candido et al. (Reference Candido, Coêlho, De Moraes and Florêncio2022) conducted a review of existing basin-level models and DSSs for public water allocation. They noted that the tools developed in the past decade tend to be prescriptive and deterministic, configured using sophisticated mathematical models for integrated economic-hydrological modelling, but with limited capabilities for spatial analysis.
The Journal of Hydrology recently dedicated a whole special issue to “Decision-support systems for water management”, motivated by the continuous development in technologies in terms of computing power, monitoring capabilities, and so on, which may lead to “a new era of water management capabilities supported by developing decision support systems (DSS)” (Wardropper and Brookfield, Reference Wardropper and Brookfield2022). The collection of papers analyses problems related to the limited uptake of DSSs by water managers and point out those related to failures in acknowledging the whole set of relevant environmental, sociocultural, and institutional factors, with a crucial role played by the limited incorporation of socioeconomic dimensions. Other widely acknowledged issues negatively affecting water DSS usability and uptake lie in the supply-side-driven development of tools, with limited involvement of stakeholders, inadequate efforts to tailor tools to institutional contexts, limited flexibility, and lack of maintenance, which add to the intrinsic complexity of issues and the diversity of application contexts (e.g. from flood risk management to real-time water allocation) and of the actors involved in the process from code developers to disciplinary experts and politicians (de Kok et al., Reference de Kok, Kofalk, Berlekamp, Hahn and Wind2009; Teodosiu et al., Reference Teodosiu, Ardeleanu and Lupu2009). De Kok et al. (Reference de Kok, Kofalk, Berlekamp, Hahn and Wind2009) also point out the issues related to the need for a methodological framework for the policy- and decision-making cycle and a conceptual model for a rational problem-solving process, taking conflicting interests into account, as a prerequisite for gaining support and thus having the opportunity to supply the information that a DSS can provide.
The MULINO project
In response to the challenges of IWRM and WFD implementation faced by water management administrations, the European Commission developed a series of implementation guidelines (EC, 2003a,b,c) and funded several research and innovation projects. One of these was called “MULti-sectoral, INtegrated and Operational Decision Support System for Sustainable Use of Water Resources at the Catchment Scale” (shortened to the MULINO project), which was funded under the Fifth RTD Framework Programme and implemented between 2001 and 2003. To deal with issues of integration and sustainability, the project consortiumFootnote 4 consisted of hydrology modellers, software developers, economists, geographers, sociologists, agronomists, and GIS specialists. The main outcome of the project was a decision support system (DSS) called MULINO-DSS (hereafter mDSS), which was developed to assist water managers in responding to the evolution of policies and management requirements by providing innovative approaches. Another primary aim was to create a tool for improving the quality of the decision-making process through the structured participation of stakeholders in order to contribute to conflict resolution by representing the goals of sustainable management in an objective and transparent way (Giupponi et al., Reference Giupponi, Mysiak, Fassio and Cogan2004; Fassio et al., Reference Fassio, Giupponi, Hiederer and Simota2005; Mysiak et al., Reference Mysiak, Giupponi and Rosato2005). Moreover, in line with the evolution of the literature, the project focused not only on the design of the tool but also on the development of a novel methodological framework called “Network analysis, System Modelling and Decision support” (NetSyMoD) to guide the decision process, with mDSS as a key component (Giupponi et al., Reference Giupponi, Sgobbi, Mysiak, Camera, Fassio, Meire, Coenen, Lombardo, Robba and Sacile2008).
NetSyMoD and mDSS proved to be useful well beyond the end of the MULINO project; thus, a long series of grants has allowed for their maintenance and continuous development for more than 20 years. Following the evolution of scientific paradigms, policies, and societal needs, the implementation contexts have moved from the original interest in WRM (Fassio et al., Reference Fassio, Giupponi, Sgobbi and Mysiak2007; Petersson et al., Reference Petersson, Giupponi and Feás2007) to other fields, such as climate change adaptation (Bojovic et al., Reference Bojovic, Bonzanigo, Giupponi and Maziotis2015; Bonzanigo et al., Reference Bonzanigo, Bojovic, Maziotis and Giupponi2015; Ceccato et al., Reference Ceccato, Giannini and Giupponi2011), agricultural development (Aleksandrova et al., Reference Aleksandrova, Gain and Giupponi2015; Bonzanigo et al., Reference Bonzanigo, Giupponi and Moussadek2016b), tourism (Bonzanigo et al., Reference Bonzanigo, Giupponi and Balbi2016a), critical infrastructures resilience (Bernhofer et al., Reference Bernhofer, Giupponi, Mojtahed, Cecconi and Campennì2019) and climate proofing of investments and planning (Giupponi et al., Reference Giupponi, Barbato, Leoni, Mercogliano, Papa, Valtorta, Zen and Zulberti2022). Figure 1 shows a timeline with the main policy references, the corresponding tool releases, a selection of applications, and the most cited papers on mDSS.
Figure 1. Evolution of mDSS development and use over time: main policy references (black); most cited papers about mDSS (red); sequence of versions released (green); selection of application contexts (blue). CCA, climate change adaptation; CEA, cost-effectiveness analysis; ES, ecosystem services; HMWB, heavily modified water bodies; PES, payment for ecosystem services.
The NetSyMoD methodological framework
NetSyMoD provides a structured and adaptable approach that facilitates the use of existing tools and the alignment of the workflow with the formal procedures employed by the responsible decision-making body in a participatory process. The main elements of the DSS-based decision process adopted by the NetSyMoD approach and supported by mDSS are reported below, while in Figure 2 (Giupponi, Reference Giupponi2014) we present the articulation in six main phases:
-
• a set of alternative options, which represent the plausible solutions to the given problem to be assessed;
-
• a set of criteria and indicators describing the main dimensions of the problem to be considered for proper decisions, for which the performance of each of the options should be estimated;
-
• a series of datasets, typically ranging from modelling outcomes to various forms of qualitative information held by experts and stakeholders, to be used for the quantification of the performance of each option with reference to the selected indicators and criteria;
-
• the identification of the different sources of uncertainty (input data, conceptualisation, future projections, etc.) and of the main exogenous drivers that may lead to diverging future scenarios;
-
• value functions, which express judgements on the ranges of the possible values of the criteria, and the objectives that should be minimised or maximised;
-
• a set of plausible future scenarios, based on which the expected performances of the proposed options must be assessed;
-
• a preference structure that defines the relative importance of different criteria in contributing to the objective function and the importance of different objectives in an overall evaluation as expressed by the stakeholders and interest groups involved through a participatory process.
Figure 2. The sequence of steps for the implementation of climate change adaptation strategies proposed by the NetSyMoD approach, in accordance with the EU CCA Guidelines. Adapted from Giupponi (Reference Giupponi2014).
As briefly described above, the application context envisaged for the NetSyMoD approach and mDSS is that of a decision required by some emerging problem or legislative requirement (the triggering factor, Phase 0 in Figure 1) to be made by a competent authority, such as a river basin district developing a programme of measures for the river basin management plan (Giupponi, Reference Giupponi2007).
First, the management problem is explored (Phase 1) and participatory activities (Phase 2) are launched through a careful stakeholder analysis, as required by the WFD (EC, 2003b), with the aims to define the decision context and identify its main actors as well as their social and power relationships. The elicitation of the actors’ views on the specific decision to be made is carried out through creative modelling workshop activities leading to the development of a shared conceptual model (i.e. a cognitive map with the main elements and relationships) and the collection of their preferences about possible solutions, decisional criteria, and their relative importance (i.e. the weights) (Feás et al., Reference Feás, Giupponi, Rosato, Pahl Wostl, Schmidt, Rizzoli and Jakeman2004).
Problem analysis and scenario development (Phase 3) are based on a conceptual model that refers to the driving force, pressure, state, impact, and response (DPSIR) framework (EEA, 1999). This conceptual model has been selected because it is widely adopted in Europe and therefore should be familiar to European policy- and decision-makers. DPSIR makes explicit the main causal relationships in human-environmental systems, their consequences for the socio-ecosystem in question, and the responses to be considered. The DPSIR causal framework thus allows the decision-maker and the involved stakeholders to explore the complex links between multi-sectoral human activities and their consequences on the environment. The use of assessment models (Phase 4) framed within the DPSIR causal framework allows the exploration of such consequences in a dynamic fashion.
Phase 5 concerns the analysis of response options to solve the problem and meet the declared objectives. It is at the core of the DSS decision analysis functionality, carried out by means of MCAM. As a result, the preferable solution to the given problem is identified and then moved to implementation (Phase 6) and possibly also to further adaptive cycles.
The mDSS software and its application environment
The mDSS tool started as a standalone piece of software, freely available on the Internet. Subsequent versions were issued as web applications, with the option of downloading the software as a standalone, offline application, which was developed for facilitating use in less-developed or remote areas where the use of web applications may be problematic. The latest version is designed as a client–server web application (see Appendix for a concise history of the various versions).
Over the years, the multilingual versions, the functionality for spatial analysis, and the coupling with system dynamic modelling were abandoned because of the effort required to maintain them in the context of evolving related software packages. Some decision rules and weighting procedures were discontinued because of limited use. Sensitivity analysis was abandoned in favour of a new component in the analysis of uncertainty that supports RDM approaches, which allows also for consideration of multiple scenarios. At present, the mDSS software is designed to facilitate the loose coupling and post-processing of model outputs to be used for MCAM and the provision of outputs that can be further processed by other software environments, such as statistical packages (see Figure 3).
Figure 3. Flow chart of the current version of the mDSS software (DPSI, driving force, pressure, state, impact; R, Response).
Problem analysis (Phase 3) is coded in the “Conceptual phase” (Simon, Reference Simon1972) of mDSS and is implemented through participatory workshops, which lead to problem structuring by means of cognitive maps and the development of a shared conceptual model. The main elements of the maps are then moved to the DPSIR page of mDSS as indicators that describe the main causal relationships within the socio-ecosystem in question and which, in turn, can be selected to quantify the criteria to be used for MCA. Responses are also defined there as the plausible options to solve the problem in question (see Figure 4).
Figure 4. The Conceptual phase in mDSS.
In the “Design phase”, the identification of the alternative response options and the selection of the decision criteria and indicators are consolidated. Model outputs and other socioeconomic and environmental information are processed to provide indicator values and stored in the analysis matrix, which presents the raw performance of every option measured with the different units and/or scales of the indicators used to quantify the selected criteria.
At the beginning of the “Choice phase”, the analysis matrix is converted into a normalised matrix – the evaluation matrix – by means of value functions that express judgements on the ranges of criteria values and convert them into non-dimensional values between 0 and 1, which represent the decision-maker’s partial utility functions (Figure 5). Normalisation is followed by weighting of the criteria, and then decision rules are applied to rank the alternative responses and identify the best option – that is, the best-performing solution according to the criteria adopted, the data collected, and the preferences expressed through the value functions and weight vectors.
Figure 5. Choice phase interface in mDSS, with the value function defined for normalising the values of the analysis matrix and building the evaluation matrix.
The decision rules available in the latest version of the mDSS software are simple additive weighting (SAW) and order weighting average (OWA) (Jiang and Eastman, Reference Jiang and and Eastman2000). SAW is the most popular decision method because of its simplicity, which makes it understandable to all participants in the decision process. It performs additive aggregation of criterion values multiplied by weights that express the criterion’s importance. OWA is slightly more complicated because it adopts a second vector of weights to be applied to the ordered weighted values of SAW. By doing so, it allows the user to control the level of trade-offs among the criteria and to represent different risk attitudes of the decision-makers – from the most optimistic behaviour (i.e. satisfaction with one or few criteria that show good performance) to the opposite situation of a risk-averse attitude, and all the cases in between. The selection of one decision rule and the completion of the required inputs in terms of weight vectors lead to the calculation of the aggregated scores and the final ranking of the alternative responses to the problem in question.
The visual investigation of the results obtained is made possible by the ranking histograms and the sustainability chart (see Figure 6), which provide a visual representation of the option performances per criterion and their balance in terms of three macro-criteria that represent the pillars of sustainability: economy, society, and environment.
Figure 6. Ranking histogram and sustainability chart, visualising how the various criteria contribute to the final ranking and how the responses perform in terms of the balance among the three pillars of sustainability to which criteria are allocated.
In participatory processes, stakeholders express different preferences in terms of valuing and weighting systems, leading to multiple implementations of the choice phase. Compromise solutions can be derived by means of group decision-making routines, namely the Borda rule and the Condorcet method (Dasgupta and Maskin, Reference Dasgupta and Maskin2008). Moreover, the uncertainty space can be explored by defining confidence intervals around each of the values used for the evaluation and adopting the preferences of multiple actors. Numerous simulations of plausible decisions can be run in parallel producing outputs that can be explored to identify the robustness of alternative solutions.
The impact of mDSS
Analysing the impact of a European Commission-funded research project such as MULINO can be challenging. Theoretically, its contribution may be measured by analysing the history of the use of its methods and DSS to support the implementation of the WFD and similar planning and decisional processes. Unfortunately, only episodic and anecdotal information is available concerning mDSS’s application outside the network of scholars involved in the initial project. Therefore, an analysis of the impact of the research has been conducted, taking into account the main articles derived from the project and considering some bibliometric indices based on the number of citations.
The first indicator of impact is the number of citations including the keywords “MULINO” and “DSS” found on Google Scholar, which is higher than 2,400. Of these, 74 are citations in scientific papers. An overview of these papers shows that most of them are about applications of the tool or are reviews of DSSs (with a focus on water management) and relevant methods (especially MCA).
A more accurate impact assessment can be done by analysing the citations of the most cited papers about mDSS: Giupponi et al. (Reference Giupponi, Mysiak, Fassio and Cogan2004), Fassio et al. (Reference Fassio, Giupponi, Hiederer and Simota2005), Mysiak et al. (Reference Mysiak, Giupponi and Rosato2005), and Giupponi (Reference Giupponi2007). These papers are included in the Scopus database, and they can be analysed with metrics offered by that platform and PlumX (Colledge, Reference Colledge2017). The results of this analysis can be then compared with the data from Web of Science (WoS) and Google Scholar, as shown in Table 1. While the number of citations of the four papers is 564 in Scopus, similar to WoS with 454, on Google Scholar this value doubles, reaching approximately 1,000. The latter includes other types of materials (e.g. presentations, blogs, and academic guides), which, in the case of a tool aimed at practical implementation, can be of interest when analysing impacts outside academic circles. Importantly, at least two of the papers have been cited in policy documents.
Table 1. Main mDSS paper citations and related bibliometric index
According to Scopus, the yearly average number of citations is around 30, with a peak above 50 around the first half of the last decade, followed by a tail of around 10 citations that persists until today.
Citation benchmarking based on SciVal’s field-weighted version of the metric for outputs in the top citation percentiles shows that the main academic papers on mDSS were in the top 4% globally when compared to the citations received by documents in the same publication year and normalised by subject area. The field-weighted citation impact for all four papers (the ratio of total citations actually received to the expected citations based on the average of the subject field) is higher than one, which indicates that the output is cited more than expected according to the global average. Through Scopus metrics, it is also possible to analyse the impact of each of the papers using its h-index, which represents the h number of citations that have been cited h times, thereby complementing the citation count with their impact. The mean value is 35, which means that at least 35 of the citations for each paper have, in turn, been cited 35 times or more.
In terms of subject area, 34.5% of the citations are from environmental science, 11.97% from computer science, 11.08% from engineering, 8.94% from agricultural and biological sciences, and 8.31% from social sciences. The geographical origin of the citations shows, as expected considering that the mDSS is the outcome of a European project, that most of the citing authors are Europeans (54.07%), followed by Asians (22.30%) and North Americans (13.04%). At the country level, it is worth noting that Chinese authors come first (10% of the total), followed by German, US, and Italian authors, each accounting for around 8% of citations.
Overall, these findings show that the MULINO project and its tool had and still have a remarkable scientific impact on water management and related research fields, with an interdisciplinary, global focus.
Lessons learned
The first lesson that all DSS developers should learn, possibly before developing and implementing their tools, is that there are no decision-makers waiting for a DSS to provide “the answer” to a water management problem. DSS tools should not try to replace decision-makers. Instead, they should focus on the improvement of the decision process by guaranteeing transparency and reproducibility, supporting collaboration within competent institutions and all relevant stakeholders, managing trade-offs, and, of course, eventually identifying plausible and acceptable solutions. As a consequence, the most important requirement for suitable DSS applications is the provision of a robust, understandable, and acceptable methodological framework within which the tools should be implemented.
Another lesson that seems evident and specific for DSS tools concerns the trade-off between ease of use and specificity on one side and flexibility and reusability on the other. At the first extreme are tools that are designed for a single case, with data, modelling tools, and decision analysis methods packaged within a single piece of software. These tools can be used only in one case, but they can be perfectly tailored to minimise the effort and increase the probability of their use by targeted users. At the other extreme are flexible tools designed for multiple purposes, thus imposing a significant burden on potential users and requiring extensive skills. In these cases, a multipurpose user interface is provided, often together with methods for decision analysis (typically MCAM) and/or generic models that can be loosely or fully coupled. These tools can be used in many different circumstances, but they require substantial effort to recognising their potential and, in many cases, for their tailoring and application. Typically, in the first case, the main actors are DSS developers working with a contract provided by user institutions, while in the latter case, DSS developers offer the tool to potential users, with an important role played by knowledge brokers and consultants that disseminate and tailor to specific applications.
There is another lesson related to the above: given the challenges inherent in their application, DSS tools are at risk of possible misuse or mystification. Again, the quality and transparency of the methodological framework, together with the skills of the professionals involved, are crucial in limiting this risk.
Other important lessons are related to information and communication technology (ICT), which develops faster than any other technology, both in terms of hardware and software. In the case of mDSS, we created the first version of mDSS as a standalone Windows desktop application for 16-bit computing systems. When computing systems changed to 32-bit architectures, we had to invest time and effort to modify the software. The development of internet applications led us to translate the software for the web, but even on the Internet, technologies change rapidly. We started the mDSS web application using Microsoft Silverlight technology, which appeared a promising avenue with many capabilities. When Microsoft abandoned Silverlight, we had to move again to newer technologies. The lesson here is that software applications have a short lifespan, and we must continuously invest time to keep the technology up to date. One dilemma here is whether this effort is possible in an academic environment or if moving to commercial development is needed. An alternative avenue may be found in open-source solutions that can attract contributions from enough developers and users in the medium-to-long term. Our DSS tool is a sort of a compromise – it has been maintained and developed by the same group of researchers, always made freely available, but the source code is not open.
Conclusions and way forward
Every decision has specific knowledge and information needs. Moreover, our decisions face increasingly complex and dynamic contexts and are affected by uncertainty; one example of this is water-management-related fields, exposed to climate change and evolving socioeconomic contexts. As a result, there is an evident growing need for scientifically sound decision support; however, the question concerning what role can be played by DSS tools is still open.
Legislation and regulations are important drivers of planning and decision-making processes, as they are for the development of DSS tools. They define objectives and constraints, identify the roles of social actors, and trigger the implementation of new procedures, which leads competent administrators to revise business-as-usual approaches. Therefore, one key criterion to judge the role played by DSS should be found in their incremental contribution to the effective implementation of evolving water management principles and paradigms.
Within normative frameworks, decision support tools have demonstrated capabilities to provide an ICT environment that facilitates the implementation of scientific knowledge in decision-making as part of participatory processes (more informed, inclusive, and transparent decision-making) and enhances the quality of decision outcomes (more effective decisions and efficient implementation). However, if intended merely as pieces of software, not embedded in a methodologically sound framework for the management of the whole decision process, these tools can do very little, and they are exposed to a high risk of misuse.
Flexibility, simplicity, and effective communication have been mentioned by many authors as prerequisites for successful DSS applications. Therein lie some of the strengths and the weaknesses of the mDSS tool, as discussed above. Throughout its development story, mDSS has lost some functionality, in particular the integration with simulation models and spatial data, while maintaining the DPSIR interface for facilitating problem exploration and public participation and the MCA functionality for the identification of preferred solutions to a given problem. Evidently, the flexibility of use and the effective communication through a rather simple interface have contributed to its survival and success, but the need to develop ad hoc modelling approaches and case-specific indicators for each new implementation have limited its use only to experienced scholars. A key role for success was also found in the help of professional facilitators of participatory processes, who contributed substantially to overcome the difficulties that often emerge when stakeholders and decision-makers are exposed to graphical and statistical information.
Twenty years of experience with mDSS allow us to identify some directions for future developments of DSS tools. They should:
-
• target existing and consolidated institutional and governance frameworks to facilitate the adoption of innovative approaches by relevant authorities;
-
• provide platforms to facilitate networking, cooperation, and the exchange of experiences, including tools, models, and data, in order to develop an open community of practice of DSS developers and users;
-
• invest in training and capacity-building activities, to facilitate dissemination, improve competences of professional facilitators, and build trust and ownership of targeted end users;
-
• develop harmonised procedures to take advantage of transnational data infrastructures;
-
• and, in general,
-
• learn from past successes and failures.
More specifically concerning mDSS, the reflections developed for this paper suggest orienting future developments towards the expansion of the MCAM decision rules and weighting procedures by including new methods; revising the website, methodological documentation, and user manual; and considering offering training programmes. New functionality will also be explored for new input/output modules designed to facilitate the adoption of the tool in conjunction with existing software packages for frequent applications such as climate change adaptation plans.
Open peer review
To view the open peer review materials for this article, please visit http://doi.org/10.1017/wat.2024.1.
Data availability statement
The mDSS software is currently available from the corresponding author upon request.
Author contribution
C.G. coordinated the design and development of mDSS and NetSyMoD as well as the writing of this paper. P.B. acted as mDSS project officer and facilitated the coordination of the research with related activities and EU policy. G.C. coordinated the coding of mDSS. J.F.V. and J.M. contributed to the design and development of the algorithms of some of the software releases. All the authors contributed to the writing of this paper.
Financial support
The grants that supported the development of mDSS and NetSyMoD are reported in the Appendix. The writing of this paper was supported by the ADAPT@VE Programme of Fondazione Eni Enrico Mattei.
Competing interest
The authors declare no conflicts of interest.
A. Appendix. Development of mDSS over 20 years
Open access
Comments
No accompanying comment.