Skip to main content Accessibility help
×
Home
Hostname: page-component-99c86f546-t82dr Total loading time: 0.392 Render date: 2021-12-08T04:23:51.948Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Research directions in policy modeling: Insights from comparative analysis of recent projects

Published online by Cambridge University Press:  13 July 2021

Alexander Ronzhyn*
Affiliation:
Institute for IS Research, University of Koblenz-Landau, Koblenz, Germany
Maria A. Wimmer*
Affiliation:
Institute for IS Research, University of Koblenz-Landau, Koblenz, Germany
*
*Corresponding authors. E-mail: ronzhyn@uni-koblenz.de and wimmer@uni-koblenz.de
*Corresponding authors. E-mail: ronzhyn@uni-koblenz.de and wimmer@uni-koblenz.de

Abstract

With the increased availability of data and the capacity to make sense of these data, computational approaches to analyze, model and simulate public policy evolved toward viable instruments to deliberate, plan, and evaluate them in different areas of application. Such examples include infrastructure, mobility, monetary, or austerity policies, policies on different aspects of societies (health, pandemic, skills, inclusion, etc.). Technological advances along with the evolution of theoretical models and frameworks open valuable opportunities, while at the same time, posing new challenges. The paper investigates the current state of research in the domain and aims at identifying the most pressing areas for future research. This is done through both literature research of policy modeling and the analysis of research and innovation projects that either focus on policy modeling or involve it as a significant component of the research design. In the paper, 16 recent projects involving the keyword policy modeling were analyzed. The majority of projects concern the application of policy modeling to a specific domain or area of interest, while several projects tackled the cross-cutting topics (risk and crisis management). The detailed analysis of the projects led to topics of future research in the domain of policy modeling. Most prominent future research topics in policy modeling include stakeholder involvement approaches, applicability of research results, handling complexity of models, integration of models from different modeling and simulation paradigms and approaches, visualization of simulation results, real-time data processing, and scalability. These aspects require further research to appropriately contribute to further advance the field.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Policy Significance Statement

Through an analysis of 16 recent projects on policy modeling in Europe and beyond, the research provides an overview regarding the area of application, relevant findings and possible future research for each project. The authors also synthesize the main directions of future research in policy modeling, providing insights into the future development of computational modeling of policy. The research is useful for policy makers as an analysis of current research and as exploration of possible future developments in the field, thereby spotting the biggest challenges and opportunities of policy modeling for data-driven and evidence-based policy decision-making.

1. Introduction

Policy modeling can be applied to make decision-making processes during the design, planning, and creation of a policy more intelligent, evidence-based, and agile (Lampathaki et al., Reference Lampathaki, Charalabidis, Osimo, Koussouris, Armenia and Askounis2011). Still, while 10 years ago, researchers voiced great optimism in computational policy modeling, tools and simulations are still not extensively used by policy makers. Hamill argues that one of the reasons for that is the complexity of the domain policy modeling (Hamill, Reference Hamill2010).

In regard to models dealing with complex social phenomena, accurate predictions are often impossible. Gilbert et al. (Reference Gilbert, Ahrweiler, Barbrook-Johnson, Narasimhan and Wilkinson2018) argue that the probabilities of certain events and directions of influence and interdependencies of different factors are the main findings of policy modeling. Still, the value of these findings can be very high: for example, learning that a certain event is in the range of the possible can help make a policy more flexible and responsive (Gilbert et al., Reference Gilbert, Ahrweiler, Barbrook-Johnson, Narasimhan and Wilkinson2018), or finding evidence that certain (desired) outcomes seem impossible may lead to the adjustment of a policy. Gilbert et al. (Reference Gilbert, Ahrweiler, Barbrook-Johnson, Narasimhan and Wilkinson2018) even argue that the process of modeling is more valuable than the actual output of the model as the modeling contributes to the understanding of complex social processes.

On the political level, policy modeling can increase political legitimacy through realizing data-driven and evidence-based decision-making: relying on data when making policy decisions (Starke and Lünich, Reference Starke and Lünich2020) and providing data confirming the effectiveness of enacted policies (Papazoglou Chalikias et al., Reference Papazoglou Chalikias, Tsampoulatidis, Tsalakanidou, Nikolopoulos, Kompatsiaris, Komninos, Doudouliakis, Papastergios, Papafilis, Karkaletsi and Chatzis2020). Another important aspect of policy modeling is the value of a model as an educational tool. Models can be useful for educating the audience and communicating how certain systems work (Voinov and Bousquet, Reference Voinov and Bousquet2010). The process of creating a model can also contribute to the understanding of phenomena by the scientists. Models, when created collaboratively, can synthesize the knowledge of many experts and provide a glimpse into the functionality of complex systems such as climate systems (Stocker et al., Reference Stocker, Qin, Plattner, Tignor, Allen, Boschung, Nauels, Xia, Bex and Midgley2013).

Analyzing and modeling a policy or creating simulation tools to run and test policy models is a significant endeavor, which requires interdisciplinary teams and collaboration among different stakeholders. Such collaborations are often realized as projects funded through external funding programs. With the increased availability of data as well as tools and technologies to deal with these data, policy modeling has evolved into an attractive and interesting research avenue to pursue. This exploratory paper aims to shed light on research needs, challenges, and opportunities in policy modeling through a literature review and analysis of recent research and development projects in the domain. The ultimate goal is to deliver a comprehensive understanding of the current developments in policy modeling. The research is motivated by the following two research questions:

RQ1: What particular contributions did/do recent research projects in policy modeling aim to deliver?

RQ2: What future research needs in the domain of policy modeling can be identified from literature review and from the analyzed research projects?

To answer the first question, research projects in policy modeling were identified and analyzed (partly in the scope of the Gov 3.0 projectFootnote 1). The analysis included 29 research projects mainly funded at EU level since 2010. For the research in this paper, the projects were further limited to 16 projects that ended in 2016 or later. For the collection of the data about the projects, a template was developed to collect the main aspects (formal data on each project, type of project, general description, technologies used, soft factors studied, technology readiness level, and pilot application area) of the projects in a comparable format. For the further analysis toward identifying research needs, a second template was developed to extract the identified research needs, the research needs the project addressed already, as well as the identified training needs and training needs addressed in the project. The analysis included a qualitative and thorough study of the deliverables of each project to distil relevant issues and distinguish the commonalities and differences between the projects. The rationale for this approach is that projects can be a useful source for future research needs of the domain for several reason: (a) joint research and piloting projects explore solutions that may be subsequently implemented on a larger scale and (b) project deliverables often discuss future research needs or future directions in the area. To find this information, the deliverables and reports of each project were analyzed for any indications such as “future directions,” “further research,” “limitations,” or similar. The authors’ discussions of these topics were summarized in the corresponding column of the second template, which is presented as an adjusted table in Section 3. As these insights are based on the findings of the project team working in the domain for several years, they are particularly valuable and shall be shared with the scholarly community.

For the second research question, a literature review, guided by (Webster and Watson, Reference Webster and Watson2002), was conducted (again initially in the Gov 3.0 projectFootnote 2 and subsequently expanded), based on keyword search in Google Scholar, Scopus, Web of Science, ACM Digital Library databases. The suitability of each article for inclusion in the reference dataset was evaluated by the researchers after perusing the article’s abstract and keywords and a quick review of the article’s full text.

With this in mind, the paper is structured as follows: Section 2 reviews relevant literature on the different aspects of policy modeling to provide the necessary foundations for the project analysis. It establishes the foundational aspects for the project description and comparison, which follows with explanations and a comparative table in Section 3. A synthesis of the information relevant for answering the research questions is provided in Section 4. In Section 5, we conclude with a summary of the research conducted and findings generated, and we spot some limitations of the research.

2. Literature Review

Ruiz Estrada defines policy modeling as “an academic or empirical research work that is supported by the use of different theories as well as quantitative or qualitative models and techniques, to analytically evaluate the past (causes) and future (effects) of any policy on society, anywhere and anytime” (Ruiz Estrada, Reference Ruiz Estrada2011, p. 524). Computational policy modeling characterizes the primarily technological means in the policy analysis and modeling, which is—compared to classical policy trials, pilots and experiments—more flexible, faster and cheaper to implement (Gilbert et al., Reference Gilbert, Ahrweiler, Barbrook-Johnson, Narasimhan and Wilkinson2018). In literature, some ambiguity exists between the terms policy modeling and computational (policy) modeling: while the former is a broader term, which, beyond technological tools, includes also nontechnological methods, instruments, and tools to generate results of a policy implementation (e.g., scenario building, serious games played without computers), the former is also used to mean the latter (i.e., policy modeling done with a computer system). In the paper at hand, we use the broader term policy modeling since our considerations embody the combination of technical and non-technical approaches to achieve more sophisticated, more informed, and more reliable policy making.

Academic research on policy modeling and on the development of technological tools provides insights into how information technology can assist policy makers. Along the policy lifecycle (Howlett and Ramesh, Reference Howlett and Ramesh1995), policy modeling can be employed on different stages: not only during the design of a policy, but also in the evaluation (or appraisal) of policies (Koliba and Zia, Reference Koliba and Zia2013; Gilbert et al., Reference Gilbert, Ahrweiler, Barbrook-Johnson, Narasimhan and Wilkinson2018). The key goal in applying comprehensive policy modeling approaches is to improve a policy or experiment with possible variations of the policy in iterative steps (HM Treasury, 2013).

The subsequent sections summarize key insights from literature on (a) different fields of policy modeling, (b) policy modeling along the policy lifecycle, (c) methods and instruments for policy modeling, and (d) different theories and paradigms of policy modeling and simulation.

2.1. Fields of policy modeling

Ruiz Estrada (Reference Ruiz Estrada2011) reviewed 1,501 papers of the Journal of Policy Making to classify the existing policy modeling research in the Journal with respect to the field in which it contributes. The author established the following twelve fields of research: (a) domestic and international trade policy modeling; (b) energy, communications, infrastructure, and transportation policy modeling; (c) environmental and natural resources management policy modeling; (d) fiscal and government spending policy modeling; (e) institutional, regulation, and negotiation policy modeling; (f) labor, employment, and population policy modeling; (g) monetary, banking, and investment policy modeling; (h) production and consumption policy modeling; (i) technological and R&D policy modeling; (j) welfare and social policy modeling; (k) economic growth and development policy modeling; and (l) miscellaneous policy modeling (Ruiz Estrada, Reference Ruiz Estrada2011).

While Ruiz Estrada already provides a sophisticated overview of fields of research of policy modeling from before 2011, his research was limited to the Journal of Policy Making and new areas may have emerged over the past decade. However, gathering a comprehensive overview would require an extensive literature study involving different databases of academic literature, which is not the objective of this paper.

2.2. Policy modeling along the policy lifecycle

Different scholars describe the policy lifecycle. Howlett and Ramesh (Reference Howlett and Ramesh1995) for example distinguish agenda setting, policy formulation, decision making, policy implementation, and policy evaluation in an iterative cycle. Agenda setting is about establishing the need for a policy or policy change. Policy Formulation includes gathering evidence from the relevant stakeholders, collecting data to better understand the political context and formulating or updating a policy. Decision making refers to the final approval by legal or strategic actors and the enactment of a policy. The policy is then implemented through its application in the relevant domain (Policy Implementation). And finally, policy evaluation looks into monitoring performance, success, impact, and acceptance of the enacted policy, which may initiate another cycle. Similar descriptions with nuances of differences can be found in Macintosh (Reference Macintosh2004). Other scholars build on one of the two models, for example, Janssen and Helbig (Reference Janssen and Helbig2018).

Policy modeling techniques can be used in each stage of the policy lifecycle (Janssen and Helbig, Reference Janssen and Helbig2018). Practically, policy modeling is most useful on the planning and evaluation stages, when modeling and simulation techniques can support the assessment of how a particular policy enactment will affect a given situational context or to extrapolate the effects of a policy over time. In both cases, models are useful to go beyond the available information and generate new knowledge, leveraging sophisticated modeling techniques (Thalheim, Reference Thalheim, Embley and Thalheim2011).

2.3. Methods and technologies for policy modeling

Policy modeling is about studying a complex policy field and trying to understand the main influence factors and how these are interacting toward particular policy objectives. Different methods can be employed to gain insight into the consequences of a particular policy. In the following, we outline several approaches discussed in literature:

A common approach is to rely on data from the past: extrapolations based on existing historical data either relating to the existing situation or a similar situation. This data-driven modeling approach is best suited for predicting repeatable situations, for example raising the taxes has been done often, so historical data can be used to reliably predict the societal consequences of the possible changes in fiscal policies. Data-driven inference is a booming strand of research as more data (big data) become available and technological capacity for the data analysis improve (Pruyt, Reference Pruyt2016). Data-driven modeling can be used for ex-ante policy evaluation, when the actual data related to the implementation of the policy is compared to the counterfactual, describing what would happen in the absence of the policy (Brandt and Dlugosch, Reference Brandt and Dlugosch2020).

A different approach is to use methods able to generate novel insights that are not directly based on historical data. This approach may be based on the opinions of experts (like in the Delphi method or in the scenario-building approach), who produce insights and ideas that go beyond extrapolation from pre-existing data. This approach is better suited for predicting the unpredictable and unexpected, also termed “unanticipated knowledge” (Zabell, Reference Zabell1992). For example, the effects of an invention of artificial intelligence (AI) may be difficult to predict based on data, because it is a unique situation that has never happened before. This area has also been transformed with the increase of computational capacity: the input from the experts can be used to build formal simulation models improving and validating the predictions (Kopec et al., Reference Kopec, Finès, Manuel, Buckeridge, Flanagan, Oderkirk, Abrahamowicz, Harper, Sharif, Okhmatovskaia, Sayre, Rahman and Wolfson2010; Stach et al., Reference Stach, Kurgan and Pedrycz2010).

In relation to these two approaches, Kim et al. (Reference Kim, Kang, Choi and Kim2017) distinguish between data modeling and simulation modeling. Simulating the different processes in society is difficult as there are a lot of variables to consider. The simulation models continue to improve and evolve over time. Simulation models often involve citizens at some point. The involvement can take different forms: from the weakest (providing feedback about the model structure or results) to the stronger like actually modifying or co-producing a model (Mureddu, Reference Mureddu2019). For the stronger forms of involvement, effectively engaging citizens is a challenge in itself: it is necessary to balance the complexity and usefulness of the feedback. If the desired feedback requires expert knowledge of the simulation model, the feedback from the citizens will be limited. While more general feedback is easier to collect, its usefulness is limited. To tackle this challenge, different approaches may be used: from supporting the simulation models with narrative scenarios and visualizations to the creation of platforms for facilitating the feedback to the specific aspects of the model (Scherer et al., Reference Scherer, Wimmer, Lotzmann, Moss and Pinotti2015; Mureddu, Reference Mureddu2019).

2.4. Theories and paradigms of policy modeling and simulation

Policy modeling and simulation can be distinguished by their underlying theories and paradigms of modeling and simulation, which started to develop as a field in the nexus of activities between government, military, and industry after World War II (see Gilbert and Troitzsch, Reference Gilbert and Troitzsch2005; Majstorovic et al., Reference Majstorovic, Wimmer, Lay-Yee, Davis, Ahrweiler, Janssen, Wimmer and Deljoo2015 for a comparative analysis of the different paradigms). Scholars differentiate Microsimulation, Agent-based Modeling, Macrosimulation, and System Dynamics.

Microsimulation is a paradigm, which mostly builds on (large) data from the past. The modeling focus is either on static (e.g., evolution of tax volume over time in a given context or territory) or dynamic factors (i.e., behavior of people towards paying tax). Bae et al. (Reference Bae, Paik, Kim, Singh and Sajjad2016), for example, describe individuals in population with micro population data and simulate their behaviors based on population statistics. Microsimulations have a long history, first mentioned in 1950s when they were used in economic modeling, they are used effectively in taxation, welfare pension and other policy areas (Birkin and Wu, Reference Birkin, Wu, Heppenstall, Crooks, See and Batty2012). If dynamic behavior is applied in microsimulation, microsimulation models can transition to the paradigm of agent-based modeling (Heppenstall et al., Reference Heppenstall, Crooks, See and Batty2012, p. 6).

Agent-based modeling (ABM) focuses on dynamic aspects/behavior of actors in complex system environments. Batty et al. (2012) argue that ABM offers “the capability of reflecting the richness of the world in a way that appears essential to any good explanation of how spatial structures such as cities, regions, the global system itself as well as all its physical components evolve and change”. ABM models the behavior of individual units (agents) involved in these processes. For example, modeling crowds by modeling the behavior of individual people. The pivotal challenge with the ABM paradigm is determining the unit of analysis; a larger number of agents leads to more complex simulation. Therefore, individual actors are often aggregated into actor groups with less granular behavioral characteristics, which bears the risk that the simulation model does no more reflect the real behavior. Scholars therefore argue that modeling the individual agents’ properties close to the reality reflects reality more accurately (Macal and North, Reference Macal and North2014; Scherer et al., Reference Scherer, Wimmer, Lotzmann, Moss and Pinotti2015).

System Dynamics is about looking at the problems on the global scale and viewing the world as a supersystem consisting of a number of complex interconnected systems (Jaeger et al., Reference Jaeger, Jansson, van der Leeuw, Resch and Tàbara2013). Simulation models based on system dynamics model a situation on a global level to describe a real-world system using analytical means via systems of differential equations (Gilbert and Troitzsch, Reference Gilbert and Troitzsch2005). A real-world system is described and analyzed as a whole on the macro-level (Forrester, Reference Forrester1961) and represented using flow diagrams and internal feedback loops (Coyle, Reference Coyle1997; Harrison et al., Reference Harrison, Lin, Carroll and Carley2007). System Dynamics models do not require much data and the output of the model consists of plots describing behavior and the changes of the initial values of the variables and parameters of the model over time. To describe the behavior of a real-world process accurately, a model needs to be run many times with different parameter values (Maria, Reference Maria1997). A typical use of system dynamics models is macro-economic modeling as well as for describing impact of policies during, for example, a spread of a disease. According to Astolfi et al. (Reference Astolfi, Lorenzoni and Oderkirk2012), system dynamics models are well suited for predicting short-term policy impacts.

It is also possible to combine different approaches within a simulation model to improve accuracy or analyze the system on different levels, realizing multilevel modeling (Majstorovic et al., Reference Majstorovic, Wimmer, Lay-Yee, Davis, Ahrweiler, Janssen, Wimmer and Deljoo2015; Bae et al., Reference Bae, Paik, Kim, Singh and Sajjad2016).

3. Overview of Projects in Policy Modeling

In order to gather insights into recent research on policy modeling (mainly in European Research), we studied 16 research projects, of which 14 are/were funded within European funding schemes and two are projects conducted outside the EU. The projects ending 2016 or more recently have been identified based on search in the funding database CordisFootnote 3 and through web search using “policy modeling” keywords and year of finalization as an exclusion criteria for the selection of the projects.

Table 1 summarizes the results of the project analysis. The template used for the analysis in this paper is based on the initial templates developed in the Gov 3.0 project and is adjusted to contain the following elements: Acronym provides the short project name along with a footnote to the project website or the Cordis database entry. Project duration lists the timespan in years, in which a project has been executed. Stage in policy lifecycle and technologies lists the stage in the policy lifecycle at which the project is centred (based on the stages presented in Section 2.2) and technologies used in the policy modeling aspect of a project. Areas list the policy modeling areas as described in Section 2.1. Projects focusing on theory development or projects providing a generic solution to “any” policy area are indicated as well. Summary outlines the focus of the project, while the Artefacts/Results lists key models/prototypes/solutions elaborated in the course of the project. Finally, the last column Identified future research lists the policy modeling-related future research directions extracted from deliverables, reports, or publications of a project.

Table 1. Overview of analyzed projects on policy modeling, which ended 2016 or later

4. Insights from the Project Analysis

In this section, we synthesize the findings of the project analysis and discuss them in relation to the existing literature on the subject.

4.1. Stages in policy lifecycle, technologies/methods, and modeling paradigm

The majority of the projects developed solutions that can be used in two or more stages of the policy lifecycle. The stages mainly addressed are agenda setting, policy formulation, and polity evaluation.

The most frequently mentioned technology is big data, used as input for simulations or models. Other technologies applied in the projects analyzed are machine learning, social data analysis, and natural language processing. The technologies used are dependent on the stage in the policy lifecycle: social media analysis is used in projects focusing agenda setting to collect inputs from citizens, while big data is more often mentioned by projects addressing policy evaluation.

Projects that involved simulations, applied either the paradigm of agent-based simulations or addressed serious games and gaming-based simulations. For example, the SYMPHONY projectFootnote 4 produced a serious game simulating the global financial market. Serious games can also be used as a modeling and prediction tool. However, often the focus of the games lies on educating the players. Agent-based games, for example, have already been used in other complex environments, like management (Jager and van der Vegt, Reference Jager and van der Vegt2015).

4.2. Domain of application and project results

The focus of the majority of the projects in the analysis is on the development of a policy simulation model of the specific domain. The main outcome of such projects is usually either a theoretical model or a prototype that is tested on specific use cases in the domain.

The areas of application, mapped to the classification presented in Section 2.1, include energy, infrastructure and transportation policy modeling [Consensus, INSIGHT, ModeRATED Policy], health policy modeling [FRESHER, REPOPA, PoliVisu, Prevention Policy Modeling Lab], financial and monetary policy modeling [SIMPOL, SYMPHONY], labor, employment and population policy modeling [COMPASS NZ], and environmental and natural resources policy modeling [SIMPOL, SUPREMA]. Some projects focused on solutions that can be applied to many different policy areas [SENSE4US, Policy Compass]. Other modeling areas include cross-cutting aspects like crisis management [Crisis] or risk modeling [RISK].

Several projects are not oriented toward modeling a specific policy issue, but rather focus on the policy modeling domain as a whole, such as Big Policy Canvas.

Most of the literature dealing with policy modeling takes a technical research path and focuses on the technologies that can support policy modeling for Government. In addition, as underlined by Brooks et al. (2018), the policy modeling strand is fragmented as it encapsulates different disciplines which are distinct from each other and meet in their own conferences and specialist fields. Realizing these limitations, scholars have attempted to strengthen or set up international communities for ICT solutions for governance and policy modeling (e.g., in the eGovPoliNet project, see Janssen et al., 2015 and Majstorovic et al., Reference Majstorovic, Wimmer, Lay-Yee, Davis, Ahrweiler, Janssen, Wimmer and Deljoo2015), aiming to build on experiences accumulated by leading actors and bringing together innovative knowledge of the field. Beyond bringing together different research communities working on policy modeling topics, their work aimed to provide a barometer for effectiveness for public governance and policy modeling in Europe and worldwide. The eGovPoliNet project provided important theoretical and empirical contribution in policy modeling research (Janssen et al., 2015). Mellouli et al. (Reference Mellouli, Shanin and Hamza2015) produced a comparative analysis of different technology frameworks for policy modeling, classifying them into three categories: (a) frameworks focusing on policy process stages, (b) frameworks focusing on institutions, and (c) those that focus on human factor and collation.

A more recent project, Big Policy Canvas (2017–2019) looked at the needs in data-driven policy making and created a roadmap of future research needs and research actions. Policy modeling was named as one of the challenge clusters in relation to data-driven policy making. Specifically, Big Policy Canvas looked into the gaps that hinder the rapid and effective uptake of policy modeling. The apparent need to accelerate the policy modeling process has been identified as one of the research gaps in the project (Mureddu, Reference Mureddu2019, p. 87). Policy modeling is also one of the six research clusters related to the big data use in public policy (Mureddu, Reference Mureddu2019), specific relevant research challenges associated with this cluster are Collaborative model simulations and scenarios generation and Integration and re-use of modeling schemes.

4.3. Future directions of research: challenges and opportunities

Summarizing the future research envisioned in the Gov 3.0 project, several trends can be formulated, which can be broadly grouped into two categories: (a) overcoming existing challenges and (b) realizing opportunities.

The challenges-related research directions include:

Stakeholder Involvement. Gilbert et al. (Reference Gilbert, Ahrweiler, Barbrook-Johnson, Narasimhan and Wilkinson2018) argue that the value of policy models will be very low, unless the decision makers are involved iteratively in the development of the policy model. It is also often necessary to involve the affected stakeholders both for improving the quality of the model and fulfilling the demand and expectation of participation (Voinov et al., Reference Voinov, Kolagani, McCall, Glynn, Kragt, Ostermann, Pierce and Ramu2016). Stakeholder involvement therefore requires common understanding (modeling standards, procedures, and methodologies; Mureddu, Reference Mureddu2019), likely the unification of modeling theories (Wimmer and Majstorovic, Reference Wimmer and Majstorovic2015) and improvements in usability of the tools [PoliVisu].

Transparency and applicability. The public demands certain level of transparency from the policy makers. Policy modeling however produces results that might be difficult to explain, policy models can be black box systems that produce certain output based on the (often very large amount of) input, while the actual way from the input to output may be very difficult to comprehend (Armenia et al., Reference Armenia, Carlini, Onori and Saullo2014). Creating more transparent models is necessary to justify policy making decisions based on them, however, this still remains a challenge [COMPASS]. At the same time there is a significant issue of public mistrust of science and the validity of scientific findings in relation to political life (Ibrahim and Larsson, Reference Ibrahim and Larsson2019; Edis, Reference Edis2020).

Complexity. As the subject of modeling is a society or its part, it is intrinsically difficult to consider all the relevant variables involved in the model: necessarily reducing the range of considered factors to a manageable amount (Easton, Reference Easton1957). Thus, the policy model would always be an imperfect attempt to simulate the reality. Models are continually becoming better with the advancements in both computing power and sophistication of the modeling approaches. Still, a number of projects listed the complexity as one of the research challenges [CRISMA, SUPREMA]. Other related significant research challenge indicated by a number of projects is the need for an integrated model in a domain. SUPREMA project report (Jongeneel et al., Reference Jongeneel, Gonzalez-Martinez, Lesschen, van Meijl, Heckelei and Salamon2020), for example, mentions the diversity of models and the challenges of maintaining them as a biggest challenge in agricultural modeling. (Mureddu et al., Reference Mureddu, Osimo, Misuraca and Armenia2012), FRESHER (Mattioli et al., Reference Mattioli, Quaranta and Vella2018), and INSIGHT (INSIGHT, 2013) projects on the other hand discussed the importance of data integration from different sources and disciplines as a way to improve the interpretation of complex data.

Visualization. Directly as a result of the previous challenge, even the most sophisticated model is useless if its users are not be able to make sense of it (Burkhardt et al., Reference Burkhardt, Nazemi, Sonntagbauer, Sonntagbauer, Kohlhammer, Wimmer, Janssen, Macintosh, Scholl and Tambouris2013). Visualizing the results of the modeling process remains a significant challenge as suggested by projects like EU Community, SYMPHONY and Big Policy Canvas (Mureddu, Reference Mureddu2019).

Comparing the identified research directions to the findings of a policy modeling roadmapping effort conducted a decade ago (Lampathaki et al., Reference Lampathaki, Charalabidis, Passas, Osimo, Bicking, Wimmer and Askounis2010), it is worth noting that fewer challenges now are purely technological, while the majority of the research findings still remain relevant even ten years on: visualization, stakeholder involvement and model integration are present both in the CROSSROAD roadmap (see Lampathaki et al., Reference Lampathaki, Charalabidis, Passas, Osimo, Bicking, Wimmer and Askounis2010) and in the current results.

The opportunities include the research directions focusing on taking advantage of the new technological developments, particularly in relations to the big data analytics, increasing computational capacities and Machine Learning algorithms (Mureddu, Reference Mureddu2019). This is particularly significant when real-time data is used [INSIGHT, PoliVisu] (INSIGHT, 2013).

Sustainability is a concept mentioned often in the project descriptions. Using policy modeling techniques is perceived as a way to reach sustainability in society or one of the domains, while sustainable models are one of the goals of policy modeling. Connected to sustainability is the need to reach wider audience with the results of policy modeling and simulations. This can be achieved through simplification of the presentation and providing visualizations of the processes and the results. Making the complex models transparent and understandable to the general public (citizens as stakeholders) is a significant challenge, not limited to the domain of policy modeling, however very often it is a challenge that needs to be addressed for the desired outcome. Connected opportunity is scalability, when the existing simulation platforms and solutions can be adapted to be used on a larger scale or in a different environment, which can also lead to significant savings (especially useful in the developing countries).

Ibrahim and Larsson (Reference Ibrahim and Larsson2019, p. 215) discuss the results of the SENSE4US project and argue that there is still “a lack of decision support tools that can provide policy-oriented modeling, simulation and decision analysis capabilities”. So, the possibilities for future work include the ways of extracting data for policy model from texts in policy reports and research literature, incorporation of information from social media, implementation of applications for stakeholder analysis (e.g., Crowdsourcing applications) (Frank, Reference Frank2013). With the significant progress in machine learning and natural language processing in relation to social media analysis, incorporating these technologies into the policy modeling solutions can be a significant opportunity to increase the accuracy and relevancy of the models.

Finally, projects such as RISK see the possibility for the application of the created model or theoretical framework in a different area or domain. This is a distinct opportunity for more theory-oriented projects, however the successful realization would often require cross-disciplinary collaborations and involvement of a wider range of stakeholders.

It is also worth discussing the research directions not identified in the projects, most notably the ethical issues. Policy models may use personal data as an input. In this regard, it is important to ensure the proper ethical standards and anonymization of data. More generally, policies affect the lives of people, and decisions based on policy modeling inform the adoption of policies. Hence, attention should be paid to avoid possible bias in the input data and ensure transparency. Thus, while absent from the results of the project analysis, ethical considerations remain an important future research direction that needs to be considered in relation to policy modeling.

5. Conclusions, limitations and further research

While policy modeling has come a long way and there have been significant advances in relation to both the development of technology and computational capacities on one hand, and approaches and theoretical policy models on the other, there are many aspects that still need further research. For example, a thorough literature review on policy modeling to complement and enrich the highly valuable research of Ruiz Estrada (Reference Ruiz Estrada2011) by extending the scope of papers beyond the Journal of Policy Making to include research from different disciplines and publication outlets is such a future research need. Such substantial research would also lead to a domain model to conceptualize and describe the breadth and depth of the research on policy modeling. Such a research could also contribute to build up a classification of types and categories of policy models used along respective modeling paradigms.

This paper introduced an analysis of 16 policy modeling projects along with literature review of the topic. Along this, four core characteristics of policy modeling have been identified (RQ1) and future research directions in policy modeling have been extracted and synthesized (RQ2).

As accurate modeling and simulation is complex, policy models are created with a specific application area in mind. And even then, the creation of a working prototype or a tool requires significant effort and funds, which often can only be realized in projects over several years, which can be seen in the most projects focusing on the specific policy area and policy-making stage. Projects that involve more than one policy area are usually more theory-oriented, not offering applied modeling or simulation solutions. Such specialization of the projects in turn sets restrictions on the re-use of the created tools and adaptation of models to other policy areas or sometimes even other geographical areas. Another related big challenge is the communication of the results of modeling, and implementing these results into real policy changes. The inherent complexity of the domain may make the communication of the findings difficult, thus limiting potential benefits of the modeling. In this regard, further research is needed on stakeholder involvement in policy development and modeling: to collect relevant information, to establish trust and to ensure that the results of policy modeling are used in real-world policy making processes. Additionally, research in visualization of the results and process of policy modeling may address further issues in relation to communicating with the key stakeholders and society as a whole.

Compared to the earlier research roadmapping efforts, technological issues seem to be less significant research targets in the analyzed projects, which shows the notable technological advances in the field of policy modeling and also changes in the focus of some of the projects. Real-time data analysis is an example, which was not included in earlier research roadmaps, but is a hot topic now with increased popularity of the smart city concept (see e.g., Nam and Pardo, Reference Nam and Pardo2011; Ojo et al., Reference Ojo, Curry, Janowski and Dzhusupova2015, p. 13; Caragliu and Del Bo, Reference Caragliu and Del Bo2019 for descriptions).

The findings of the analysis in the paper are not comprehensive and some of the less evident aspects or research needs of the policy modeling domain were likely not identified due to the limitations of the research methodology. Still, the number and variety of the projects in the analysis allowed to identify the major future research directions in the field and can serve as a useful overview of the state of research in policy modeling.

Another limitation of this research is that, while the effort was made to provide as complete picture of existing policy modeling projects, it is possible that some projects were not included in the analysis. For some projects, the deliverables and final reports were not readily available, which produced gaps in the table. Future research may include an in-depth comparison of the models and simulation approaches designed or discussed in the projects similar to the approach taken by Majstorovic et al. (Reference Majstorovic, Wimmer, Lay-Yee, Davis, Ahrweiler, Janssen, Wimmer and Deljoo2015). Alternatively, the research can be extended by using papers as a unit of analysis, as many more models and approaches are not directly connected to a specific project, but result from the ongoing research at the research groups and centers.

Funding Statement

The research partially based on the data collected for the Deliverable 2.2 (Roadmap) of the Gov 3.0 project. The Gov 3.0 project was funded by the Erasmus+ Knowledge Alliance, Project Reference No. 588306-EPP-1-2017-1-EL-EPPKA2-KA.

Competing Interests

The authors declare no competing interests exist.

Data Availability Statement

Data availability is not applicable to this article as no new data were created or analysed in this study.

Author Contributions

Conceptualization: M.W.; A.R. Formal analysis: A.R; M.W. Funding acquisition: M.W. Investigation: A.R; M.W. Methodology: M.W; A.R. Supervision: M.W; Writing original draft: A.R, M.W; Writing-review & editing: M.W; A.R. All authors approved the final submitted draft.

References

Armenia, S, Carlini, C, Onori, R and Saullo, AP (2014) Policy Modeling as a new area for research: Perspectives for a systems thinking and system dynamics approach? 2nd International Symposium “System Thinking for a Sustainable Economy”, pp. 1–22.Google Scholar
Astolfi, R, Lorenzoni, L and Oderkirk, J (2012) A Comparative Analysis of Health Forecasting Methods. OECD Health Working Papers, No. 59. Paris: OECD Publishing.Google Scholar
Bae, JW, Paik, E, Kim, K, Singh, K and Sajjad, M (2016) Combining microsimulation and agent-based model for micro-level population dynamics. Procedia Computer Science 80, 507517. https://doi.org/10.1016/j.procs.2016.05.331CrossRefGoogle Scholar
Batty, M (2012) Agent-Based Models of Geographical Systems. Springer Netherlands, Dordrecht.CrossRefGoogle Scholar
Birkin, M and Wu, B (2012) A review of microsimulation and hybrid agent-based approaches. In Heppenstall, AJ, Crooks, AT, See, LM, Batty, M (eds), Agent-Based Models of Geographical Systems. Berlin: Springer. https://doi.org/10.1007/978-90-481-8927-4_3Google Scholar
Brandt, T and Dlugosch, O (2020) Exploratory data science for discovery and ex-ante assessment of operational policies: Insights from vehicle sharing. Journal of Operations Management 67(3), 307328. https://doi.org/10.1002/joom.1125CrossRefGoogle Scholar
Brooks, L., Janssen, M., & Papazafeiropoulou, A. (2018). eGovPoliNet: Experiences from Building a Policy Informatics Research Community. In Policy Analytics, Modelling, and Informatics (pp. 39-75). Springer, Cham.CrossRefGoogle Scholar
Burkhardt, D, Nazemi, K, Sonntagbauer, P, Sonntagbauer, S and Kohlhammer, J (2013) Interactive visualizations in the process of policy modeling. In Wimmer, MA, Janssen, M, Macintosh, A, Scholl, HJ and Tambouris, E (eds), Electronic Government and Electronic Participation—Joint Proceedings of Ongoing Research of IFIP EGOV and IFIP ePart 2023. Bonn: Gesellschaft für Informatik e.V., pp. 104115.Google Scholar
Caragliu, A and Del Bo, CF (2019) Smart innovative cities: The impact of Smart City policies on urban innovation. Technological Forecasting and Social Change 142, 373383.CrossRefGoogle Scholar
Coyle, RG (1997) System dynamics modelling: A practical approach. Journal of the Operational Research Society 48(5), 544. https://doi.org/10.1057/palgrave.jors.2600682CrossRefGoogle Scholar
Easton, D (1957) An approach to the analysis of political systems. World Politics 9(3), 383400. https://doi.org/10.2307/2008920CrossRefGoogle Scholar
Edis, T. (2020) A revolt against expertise: Pseudoscience, right-wing populism, and post-truth politics. Disputatio – Philosophical Research Bulletin 9(13). https://doi.org/10.5281/zenodo.3567166Google Scholar
Forrester, J (1961) Industrial Dynamics. Cambridge, MA: MIT Press.Google Scholar
Frank, S (2013) D3.2.1 Models and Simulators Report. Retrieved January 21, 2021, from https://cordis.europa.eu/project/rcn/110507Google Scholar
Gilbert, N, Ahrweiler, P, Barbrook-Johnson, P, Narasimhan, KP and Wilkinson, H (2018) Computational modelling of public policy: Reflections on practice. JASSS, 21(1), 14. https://doi.org/10.18564/jasss.3669CrossRefGoogle Scholar
Gilbert, N and Troitzsch, K (2005) Simulation for the Social Scientist. London: McGraw-Hill Education.Google Scholar
Hamill, L (2010) Agent-based modelling: The next 15 years. JASSS 13(4), 17. https://doi.org/10.18564/jasss.1640CrossRefGoogle Scholar
Harrison, JR, Lin, Z, Carroll, GR and Carley, KM (2007) Simulation modeling in organizational and management research. Academy of Management Review 32(4). https://doi.org/10.5465/AMR.2007.26586485CrossRefGoogle Scholar
Heppenstall, AJJ, Crooks, AT, See, LM and Batty, M (2012) Agent-based models of geographical systems. In Agent-Based Models of Geographical Systems. Berlin: Springer. https://doi.org/10.1007/978-90-481-8927-4CrossRefGoogle Scholar
HM Treasury (2013) The GreenBook: Appraisal and Evaluation in Central Government. Retrieved January 21, 2021, from https://www.gov.uk/government/publications/the-green-book-appraisal-and-evaluation-in-central-governentGoogle Scholar
Howlett, M and Ramesh, M (1995) Studying Public Policy: Policy Cycles and Policy Subsystems. Oxford: Oxford University Press. https://doi.org/10.1017/S0008423900007423Google Scholar
Ibrahim, O and Larsson, A (2019) Intelligibility and transparency in model-based collaborative governance. ACM International Conference Proceeding Series. https://doi.org/10.1145/3325112.3325247CrossRefGoogle Scholar
INSIGHT (2013) Policy Modelling and Governance Tools for Sustainable Urban Development State-of-the-Art and Future Challenges (Issue October). Retrieved January 21, 2021, from http://www.nommon-files.es/working_papers/INSIGHT_PositionPaper_Oct2013.pdfGoogle Scholar
Jaeger, C, Jansson, P, van der Leeuw, S, Resch, M and Tàbara, JD (2013) GSS: Towards a Research Program for Global Systems Science. In: Second Open Global Systems Science Conference. pp 127.Google Scholar
Jager, W and van der Vegt, G (2015) Management of complex systems: Toward agent-based gaming for policy. In Public Administration and Information Technology. Berlin: Springer. https://doi.org/10.1007/978-3-319-12784-2_13Google Scholar
Janssen, M., Wimmer, M. A., & Deljoo, A. (Eds.). (2015). Policy practice and digital science: Integrating complex systems, social simulation and public administration in policy research (Vol. 10). Springer.CrossRefGoogle Scholar
Janssen, M and Helbig, N (2018) Innovating and changing the policy-cycle: Policy-makers be prepared! Government Information Quarterly 35(4), S99S105. https://doi.org/10.1016/j.giq.2015.11.009.CrossRefGoogle Scholar
Jongeneel, R, Gonzalez-Martinez, A, Lesschen, JP, van Meijl, H, Heckelei, T and Salamon, P (2020) Deliverable 1.10: The SUPREMA Roadmap exploring future directions for agricultural modelling in the EU. Project Support for Policy Relevant Modelling ofAgriculture (SUPREMA). Retrieved January 20, 2021, from https://www.suprema-project.eu/images/RoadMap-Conclusions-PolicyBrief1.pdfGoogle Scholar
Kim, BS, Kang, BG, Choi, SH and Kim, TG (2017) Data modeling versus simulation modeling in the big data era: Case study of a greenhouse control system. Simulation 93(7). https://doi.org/10.1177/0037549717692866CrossRefGoogle Scholar
Koliba, C and Zia, A (2013) Governance informatics: Using computer simulation models to deepen situational awareness and governance design considerations. In Governance in the Information Era (pp. 207230). Routledge.Google Scholar
Kopec, JA, Finès, P, Manuel, DG, Buckeridge, DL, Flanagan, WM, Oderkirk, J, Abrahamowicz, M, Harper, S, Sharif, B, Okhmatovskaia, A, Sayre, EC, Rahman, MM and Wolfson, MC (2010) Validation of population-based disease simulation models: A review of concepts and methods. BMC Public Health. https://doi.org/10.1186/1471-2458-10-710CrossRefGoogle ScholarPubMed
Lampathaki, F, Charalabidis, Y, Osimo, D, Koussouris, S, Armenia, S and Askounis, D (2011) Paving the way for future research in ICT for governance and policy modelling. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22878-0_5Google Scholar
Lampathaki, F, Charalabidis, Y, Passas, S, Osimo, D, Bicking, M, Wimmer, MA and Askounis, D (2010) Defining a taxonomy for research areas on ICT for governance and policy modelling. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14799-9_6Google Scholar
Macal, C and North, M (2014) Introductory tutorial: Agent-based modeling and simulation. In Proceedings of the Winter Simulation Conference 2014, pp. 6–20.CrossRefGoogle Scholar
Macintosh, A (2004) Characterizing e-participation in policy-making. In Proceedings of the Hawaii International Conference on System Sciences. https://doi.org/10.1109/hicss.2004.1265300.CrossRefGoogle Scholar
Majstorovic, D, Wimmer, MA, Lay-Yee, R, Davis, P and Ahrweiler, P (2015) Features and added value of simulation models using different modelling approaches supporting policy-making: A comparative analysis. In Janssen, M, Wimmer, MA and Deljoo, A (eds), Policy Practice and Digital Science: Integrating Complex Systems, Social Simulation and Public Administration in Policy Research. Berlin: Springer International Publishing, pp. 95124.CrossRefGoogle Scholar
Maria, A (1997) Introduction to modeling and simulation. In Proceedings of the 29th Conference on Winter Simulation, pp. 7–13.CrossRefGoogle Scholar
Mattioli, B, Quaranta, MG and Vella, S (2018) D6.4 Synthesis Document: Future Public Health Research Agenda. Available at https://www.foresight-fresher.eu/content/uploads/2018/03/d6-4-31-01-2018.pdfGoogle Scholar
Mellouli, S, Shanin, J and Hamza, K (2015) Comparative Analysis of Technology Frameworks. Koblenz: eGovPoliNet: The Policy Community.Google Scholar
Mureddu, F (2019) D5.2 Roadmap for Future Research Directions –Big Policy Canvas Project. Brussels: Lisbon Council.Google Scholar
Mureddu, F, Osimo, D, Misuraca, G and Armenia, S (2012). A new roadmap for next-generation policy-making. In ACM International Conference Proceeding Series – ICEGOV 2012, pp. 6266. https://doi.org/10.1145/2463728.2463743CrossRefGoogle Scholar
Nam, T and Pardo, TA (2011) Smart city as urban innovation: Focusing on management, policy, and context. In Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance. New York: ACM Press, pp. 185194CrossRefGoogle Scholar
Ojo, A, Curry, E, Janowski, T and Dzhusupova, Z (2015) Designing next generation Smart City initiatives: The SCID framework. In Transforming City Governments for Successful Smart Cities. Cham: Springer International Publishing, pp. 4367.CrossRefGoogle Scholar
Papazoglou Chalikias, A, Tsampoulatidis, I, Tsalakanidou, F, Nikolopoulos, S, Kompatsiaris, I, Komninos, N, Doudouliakis, K, Papastergios, G, Papafilis, P, Karkaletsi, S and Chatzis, C (2020) Evidence-driven policy-making using heterogeneous data sources—The case of a controlled parking system in Thessaloniki. Data & Policy 2: e15. https://doi.org/10.1017/dap.2020.15.CrossRefGoogle Scholar
Pruyt, E (2016) Integrating systems modelling and data science: The joint future of simulation and “big data” science. In Artificial Intelligence: Concepts, Methodologies, Tools, and Applications (pp. 822840). IGI Global. https://doi.org/10.4018/978-1-5225-1759-7.ch033Google Scholar
Ruiz Estrada, MA (2011) Policy modeling: Definition, classification and evaluation. Journal of Policy Modeling 33: 523536. https://doi.org/10.1016/j.jpolmod.2011.02.003CrossRefGoogle Scholar
Scherer, S, Wimmer, M, Lotzmann, U, Moss, S and Pinotti, D (2015) Evidence based and conceptual model driven approach for agent-based policy modelling. J Artif Soc Soc Simul 18(3): 14.https://doi.org/10.18564/jasss.2834CrossRefGoogle Scholar
Stach, W, Kurgan, L and Pedrycz, W (2010) Expert-based and computational methods for developing fuzzy cognitive maps. Studies in Fuzziness and Soft Computing. https://doi.org/10.1007/978-3-642-03220-2_2CrossRefGoogle Scholar
Starke, C and Lünich, M (2020) Artificial intelligence for political decision-making in the European Union: Effects on citizens’ perceptions of input, throughput, and output legitimacy. Data & Policy 2: e16. https://doi.org/10.1017/dap.2020.19CrossRefGoogle Scholar
Stocker, TF, Qin, D, Plattner, GK, Tignor, MMB, Allen, SK, Boschung, J, Nauels, A, Xia, Y, Bex, V and Midgley, PM (2013) Climate change 2013 the physical science basis: Working group I contribution to the fifth assessment report of the intergovernmental panel on climate change. In Climate Change 2013 the Physical Science Basis: Working Group I Contribution to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. https://doi.org/10.1017/CBO9781107415324CrossRefGoogle Scholar
Thalheim, B (2011) The theory of conceptual models, the theory of conceptual modelling and foundations of conceptual modelling. In Embley, D, Thalheim, B (eds). Handbook of Conceptual Modeling. Berlin, Heidelberg: Springer, pp. 543577. https://doi.org/10.1007/978-3-642-15865-0_17CrossRefGoogle Scholar
Voinov, A and Bousquet, F (2010) Modelling with stakeholders. Environmental Modelling and Software 25(11): 12681281. https://doi.org/10.1016/j.envsoft.2010.03.007CrossRefGoogle Scholar
Voinov, A, Kolagani, N, McCall, MK, Glynn, PD, Kragt, ME, Ostermann, FO, Pierce, SA and Ramu, P (2016). Modelling with stakeholders—next generation. In Environmental Modelling and Software 77: 196220. https://doi.org/10.1016/j.envsoft.2015.11.016CrossRefGoogle Scholar
Webster, J and Watson, RT (2002) Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly 26(2), xiiixxiii.Google Scholar
Wimmer, MA and Majstorovic, D (2015) Final report on knowledge assets in portal and final grand challenges (D 4.3). eGovPoliNet—the Policy Community. Retrieved January 21, 2021, from http://www.egov.iwvi.uni-koblenz.de:8080/eGovPoliNet/results/public-deliverables/d-4.3-final-report-on-knowledge-assets-in-portal-and-final-grand-challengesGoogle Scholar
Zabell, SL (1992) Predicting the unpredictable. Synthese 90(2): 205232. https://doi.org/10.1007/BF00485351CrossRefGoogle Scholar
Figure 0

Table 1. Overview of analyzed projects on policy modeling, which ended 2016 or later

Submit a response

Comments

No Comments have been published for this article.
You have Access
Open access

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Research directions in policy modeling: Insights from comparative analysis of recent projects
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Research directions in policy modeling: Insights from comparative analysis of recent projects
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Research directions in policy modeling: Insights from comparative analysis of recent projects
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *