Policy Significance Statement
This research helps urban mobility policymakers to get more insights in how to better use data in the policymaking process, which provides new opportunities for policymakers toward evidence-based policymaking. In this context, it answers the questions “Is big data increasingly employed and found more useful and powerful than traditional survey data?” “How are the new types of data applied in mobility policy practices” that policymakers usually have when they apply data in urban mobility policy evaluations. Based on the analysis of 74 cases, we summarize the strengths and limitations of different types of data and provide policymakers recommendations for how the potential use of data in sustainability assessment of urban mobility policy and decision-making can be better understood and tapped.
1. Introduction
Cities around the globe struggle to create better and more equitable access to important destinations and services, all the while reducing the energy consumption and environmental impacts of mobility (Schiller and Kenworthy, Reference Schiller and Kenworthy2017). Urban mobility issues are crucial problems in many regions because of rapid urbanization in the last several decades, which puts significant pressure on environmental quality, economic structure, and public health in urban areas, and challenges mobility policies (Fedra, Reference Fedra2004). It is now almost three decades since the concept of “sustainable mobility” first appeared in the 1992 EU Green Paper on the Impact of Transport on the Environment. Nevertheless, the transport sector still consumes approximately one-third of our final energy and probably causes more environmental and social problems than any other sector (Holden et al., Reference Holden, Gilpin and Banister2019). Although much progress in understanding its “unsustainabilities” has been made (Gwilliam et al., Reference Gwilliam, Kojima and Johnson2004; Cepeda et al., Reference Cepeda, Schoufour, Freak-Poli, Koolhaas, Dhana, Bramer and Franco2017; Forehead and Huynh, Reference Forehead and Huynh2018), this has not yet led to the implementation of corresponding policies in practice, leaving urban mobility systems still far from sustainable (European Commission, 2021).
In order to improve the effectiveness of policies, there is need for more evidence-based policymaking (Howlett and Giest, Reference Howlett and Giest2012). Evidence-based policymaking requires ex ante assessment of policies, based on data and sound methods. Typical challenges for the effective monitoring and evaluation in urban policymaking practice are: limited financial and staff resources; gaps in technical knowledge; and experience with regard to defining performance indicators, the retrieval, collection, preparation, and interpretation of data (Gühnemann, Reference Gühnemann2016). One of the challenges the EU’s regulatory scrutiny board has highlighted in the 5th international conference Data for Policy 2020 is the problem of a lack of data: the necessary data in order to evaluate the impact of the policy. Moreover, earlier studies also found that a lack of data and a poor culture of conducting monitoring and evaluation activities in urban governments are limitations in policymaking practice (Chinellato et al., Reference Chinellato, Koska and Werland2017; Awasthi et al., Reference Awasthi, Omrani and Gerber2018). From interviews with cities that are relatively advanced with sustainable mobility planning, it emerged that for many relevant indicators, data availability and use are restricted—data are either not available at all, its use is restricted, or there is a fee for doing so (Chinellato et al., Reference Chinellato, Koska and Werland2017). Additionally, many cities do not have experience with conceptualizing and conducting evaluations and selecting the most appropriate indicators (Chinellato et al., Reference Chinellato, Koska and Werland2017).
At the same time, developments in the last decade regarding the availability and openness of (big) data seem to provide new opportunities for evidence-based policymaking. Open data are touted as having the potential to transform science and fast-track the development of new knowledge (Dietric et al., Reference Dietrich, Gray, McNamara, Poikola, Pollock, Tait and Zijlstra2009). Urban data centers are emerging (Statistics Netherlands (CBS), 2019), while the UN has organized the first UN World Data Forum. The improved access to both traditional and new types of data have the potential to improve evidence-based evaluations of policies regarding sustainability. But how this new potential can be tapped in policy practice is an emerging problem faced by the urban mobility policymakers (OECD, 2016).
Although data may not necessarily be a blessing for policy evaluations, big data is increasingly employed and found more useful and powerful than traditional survey data. Still, it is yet unclear how the new type of data can be applied best in mobility policy practices, for instance, in which part of practical policy cycles. This paper seeks to answer such questions, which helps urban mobility policymakers to get more insights on how to better use data in the policymaking process and also provides new opportunities for policymakers toward evidence-based policymaking. It reviews the state of the art of data use in sustainability assessment (SA) of urban mobility policy in academic literature. Based on the review, this research gives insights on how different types of data are used in urban mobility policy assessment and provides recommendations about how to tap potential for evidence-based policymaking.
The paper is structured as follows. After describing the policy domain of study, urban mobility policymaking and SA are explained in more detail in Section 2. After that, we describe our research method in Section 3. Then, we classify the various types of data used in SAs of urban mobility policies and transportation management (Section 4). In Section 5, we describe a review of 74 case studies to show how these types of data are employed in different (academic) urban mobility evaluations and discuss the advantages and disadvantages of them. Based on an analysis of these cases, we discuss how to improve data use in SA of urban mobility policies. Finally, Section 7 concludes.
2. Urban Mobility and Sustainable Assessment
2.1. Urban mobility
Urban mobility refers to the “way people move in urban areas,” considering all transportation modes (De Oliveira Cavalcanti et al., Reference De Oliveira Cavalcanti, Limont, Dziedzic and Fernandes2017). As noted, urban planners are challenged to keep urban areas accessible in an equitable and resource efficient way amidst the challenges regarding rapid urbanization, climate change, and others. Under such pressure, traditional urban mobility planning is struggling to give weight to sustainability in policymaking and project implementing, and to adapt to the continually changing social circumstances. Most of the traditional transportation modes consume considerable amounts of energy and resources, which mainly focus on efficiency and convenience for travelers but are highly depended on unrecycled materials and cause serious environmental pollution with negative effects on human health (Schiller and Kenworthy, Reference Schiller and Kenworthy2017).
It is now almost three decades since the concept of “sustainable mobility” first appeared in the 1992 EU Green Paper on the Impact of Transport on the Environment (Com, Reference Com1992). In 1990, the belief that urban mobility was not sustainable as it was developing became more mainstream among local governments in Europe (Com, Reference Com1992). The need for a different approach was seen that included much more priority on public transportation. Nevertheless, cars were still given great freedom, although somewhat restrained by parking limitations and charges, sometimes justified by environmental reasons (Holden et al., Reference Holden, Gilpin and Banister2019). This approach was seen to be more “balanced” as the case was made that the car had to adapt to the city and that the city could no longer cope with the congestion that resulted from the continued growth in car use (Holden et al., Reference Holden, Gilpin and Banister2019). Next to the promotion of public transport, technology was introduced to manage demand to use existing infrastructure most optimally (e.g., traffic control systems, parking indicator systems, and traffic free central areas).
At the beginning of the century, urban mobility had still not become more sustainable (Holden et al., Reference Holden, Gilpin and Banister2019). Although much had been learned about the nature of the problem in a technical sense including possible solutions, the barriers to implement changes in practice had not been overcome (Costa et al., Reference Costa, Neto and Bertolde2017; Ellis and Glover Reference Ellis and Glover2019). This sheds light on the societal complexity of the problem: the idea that solutions can be implemented top-down is incorrect, but solutions need to be co-created with multiple actors, transport and parking operators, citizens, businesses, NGOs, along with the municipality.
The 2011 White Paper acknowledged that “still, the transport system is not sustainable” (EU, 2011 p. 4), and stated that “curbing mobility is not an option” (EU, 2011 p. 5). Instead, the White Paper called for a common strategy of de-carbonization.
Since the adoption of the European Commission’s Urban Mobility Package in 2013, the Sustainable Urban Mobility Plan (SUMP) concept has been promoted as a strategic planning instrument for local authorities. It has been proposed as a framework to foster the balanced development and integration of all transport modes and create a harmonized transport offer, while also encouraging a shift toward more sustainable modes and improving transport accessibility for all.
An “Urban Agenda” for the EU was launched in May 2016. It represents a new multilevel working method promoting cooperation between Member States, cities, the European Commission and other stakeholders in order to stimulate growth, livability, and innovation in the cities of Europe and to identify and successfully tackle social challenges. It includes a section on urban mobility, Partnership for Urban Mobility (PUM, EU, 2018), which proposes solutions to improve the framework conditions for urban mobility for cities across the EU. This covers issues relevant to technological advancements, encouraging the use of active modes of transport, improving public transport, and promoting multilevel governance measures.
Based on a survey across 328 cities in Europe in 2017, 44% said they are already conducting integrated sustainability transport planning, including 37% which said they have a plan that qualifies as a SUMP (as defined above). In addition, 16% of cities surveyed declared they were currently developing a SUMP, while 19% were eager to do so. There is a clear growth of cities with well-established SUMP’s from 7 in 2011 to 19 in 2017 (Chinellato et al., Reference Chinellato, Koska and Werland2017). The study also states that simply making SUMPs obligatory in itself does not guarantee the adoption of good quality SUMPs (Chinellato et al., Reference Chinellato, Koska and Werland2017, p. 17). Hence, it is “the way in which” the SUMP is developed and implemented that makes it effective or not. If the local political will and majority for transformation toward low-carbon mobilities is not present, a SUMP plan is unlikely to have much effect.
In summary, after 2010 attention for more structural changes in urban mobility is growing (e.g., modal shift from car mobility to other modes, associated to more attention for public health and livability), and the Paris Agreement has given further thrust to this trend. The question is to which extent these more structural changes are occurring. At first glance, it seems that despite much attention for “sustainable mobility,” both at EU, national and local level, the modal share of car mobility in urban areas is not decreasing significantly. In various urban areas the concept of “sustainable mobility” is reduced to promoting electric mobility and cleaner fuel, but not car alternatives (Bi et al., Reference Bi, Kan, Mi, Zhang, Zhao and Keoleian2016; Calise et al., Reference Calise, Cappiello, Cartenì, d’Accadia and Vicidomini2019).
2.2. Sustainability assessment
According to the research of Intelligent Energy Europe, an EU Program, there are four key policy (making) challenges for sustainable urban mobility: participation, cooperation, measure selection, and monitoring and evaluation (European Commission, 2016). Monitoring and evaluation is a key process to decide if the policies and plans could be implemented in further steps and which measures or approaches should be improved according to the results of evaluations (Chandrakumar and McLaren, Reference Chandrakumar and McLaren2018). Measure selection should be based on ex ante assessment of options.
SA is an important tool to do such ex ante policy evaluations in an integrated way. SA has been regarded as a “marriage” between environmental assessment and sustainable development (Dijk et al., Reference Dijk, de Kraker, van Zeijl-Rozema, van Lente, Beumer, Beemsterboer and Valkering2017). It refers to the systematic and integrated frameworks to assess and identify the effects of alternative undertakings and find the best way for progress toward sustainability (Pope et al., Reference Pope, Annandale and Morrison-Saunders2004; Gibson et al., Reference Gibson, Hassan and Tansey2013). It has been widely used in the sustainability evaluation of urban mobility policies (Lima et al., Reference Lima, da Silva Lima and da Silva2014; De Oliveira Cavalcanti et al., Reference De Oliveira Cavalcanti, Limont, Dziedzic and Fernandes2017).
SA, like policy assessment and formulation, generally consists of four steps (De Ridder et al., Reference De Ridder, Turnpenny, Nilsson and Von Raggamby2007): (a) problem analysis, (b) finding options, (c) assessment of options, and (d) follow-up. Ideally, the problem analysis involves data-based evaluation, as Jordan and Turnpenny (Reference Jordan and Turnpenny2015) note:
“Having established the existence of a policy problem (or problems) through some form of data collection, the various policy-relevant dimensions of the problem are then evaluated to determine their causes and extent, chiefly as a basis for identifying potential policy solutions. (…) While the point is often made that causation tends to be difficult to precisely establish, Wolman observes that “the better the understanding is of the causal process … the more likely … we will be able to devise public policy to deal with it successfully” (Wolman Reference Wolman1981, p. 437). Understanding causation, as Wolman puts it, is also reliant on the generation of adequate theoretical propositions in addition to relevant data on which to support them.”
Clearly, data are vital element of both ex ante assessment of measures and also of monitoring and evaluation in order to understand the current urban mobility status, including the role of implemented policy (Keseru et al., Reference Keseru, Wuytens and Macharis2019). In practice in Europe however, as noted in Section 1, policy evaluation is generally rather limited and a lack of data and a poor culture of conducting monitoring and evaluation activities exists in urban governments (Chinellato et al., Reference Chinellato, Koska and Werland2017). The rest of this paper seeks to review academic literature to sketch the state-of-the-art on the role of data in SA of urban mobility policy.
3. Materials and Methods
In order to understand the current state-of-the-art data use in urban mobility policy assessments and further to explore the potentials of different types of data applied in this process, we used systematic and critical review as a method to search the relevant published academic books, journal papers, and governmental documents that reported on them, available in academic databases. The whole process is depicted in Figure 1, followed the guidelines of Liberati et al. (Reference Liberati, Altman, Tetzlaff, Mulrow, Gøtzsche, Ioannidis and Moher2009).
Information flow of literature search and review.
The search term “(urban AND [mobility OR transport OR travel] AND policy AND data)” existed in title–abstract–keywords fields was used in Scopus and Web of Science. The date parameters of publication were limited to 2000–2020 and the search inspected all records published until 1st July 2020. The search in Web of Science led to 2266 records and the search in Scopus led to 2,882 records, of which 647 books, chapters, articles not in English, or not peer-reviewed were removed. Then 4,501 articles were screened by title and abstract. After, these literatures were eye-balled to remove duplicates and the articles are not consistent with the search keywords. The number of literatures then cut to 2,544.
In terms of selecting the studies that are relevant for this research in the full-text read process, the criteria for the selection is presented in Table 1. By reviewing and understanding the data use in these literatures, we identify four different types of data that are frequently used in urban mobility studies and policymaking processes: survey data, statistical data, Geographic Information System (GIS) data, and big data, which has been illustrated more in details in part 4. Hundred and sixty-six papers met the criteria were selected for a second-round of full-text screening for eligibility. Finally, 74 of them were reviewed and analyzed in the case studies (see Table 2 and more details about these cases are list in Appendix A). These final selected cases give most extensive insights about data use in their studies as well as show the state of art of how the data promote or impede policy evaluations. For the discussion part of this review, we also refer to other papers that are not included in the systematic review to discuss findings and for critical review.
Literature selection criteria
Policy-associated process in the literatures
4. Classification of Data Use in Urban Mobility Studies
Data have played a role in mobility policy planning for decades (Meyer, Reference Meyer2016) and forecasted travel demand, often based on extrapolation from historic traffic intensities, has been important. Also, household travel surveys have been a typical way to understand travelers’ behavior and to evaluate specific mobility policy (Chen et al., Reference Chen, Ma, Susilo, Liu and Wang2016). Analyzing previous governmental statistical yearbooks and relevant policies, building spatial transportation models, as well as collecting commuters’ daily travel data, are the key approaches to study the most important mobility issues including travel safety, transport system design, and sustainable mobility development (Hall, Reference Hall2012). More recently, big data has emerged in mobility studies, which has been largely used in road user’s behavior detections and travel modal shift operations (Welch and Widita, Reference Welch and Widita2019). Across these data applications, we can identify four different types of data that are frequently used in urban mobility studies and policymaking processes: survey data, statistical data, GIS data, and big data.
4.1. Survey data
Survey data have been widely used in various research domains (i.e., social science, economics, policy assessments, and risk management). In urban mobility research, they have largely applied through travel surveys to analyze the motivations and reasons of traveler behaviors in order to stimulate more sustainable mobility behaviors (Bamberg et al., Reference Bamberg, Rölle and Weber2003; Cao et al., Reference Cao, Mokhtarian and Handy2008; Long and Thill, Reference Long and Thill2015). It is also a new trend to combine Global Positioning System (GPS), smart cards, and such kind of sensing data with survey data together to comprehensively understand urban mobility issues in different angles. These combinations give researchers more opportunities to upscale their studies (Long and Thill, Reference Long and Thill2015; Gong et al., Reference Gong, Liu, Wu and Liu2016).
However, some critics argue that survey data are often constrained by unrepresentative sample sizes. For example, the household surveys by the Federal Highway Administration in the United States had a relatively small sample size compared to the size of the project (TMIP, 2013). Furthermore, it may also lose the representativeness if investigators choose unsuitable survey targets. Thus, it is critical to give a certain range of which sort of projects or researches are fitted to use survey data for their studies.
4.2. Statistical data
Statistical data in this study are defined as the statistics compiled from statistical yearbooks and various related documents, which are normally sourced from data collection by official departments and published in governmental reports. It is significant to use statistical information for understanding and quantifying impacts of political decisions in a specific area, which also plays an important role in different research domains, especially in projects, policies, and social development evaluations (Huo et al., Reference Huo, Ren, Zhang, Cai, Feng, Zhou and Wang2018; Liu et al., Reference Liu, Liu, Chen, Liu and Deng2018; Yang, Reference Yang2018).
In urban mobility studies, statistical data have been widely used by policymakers to assess mobility status and to evaluate the implemented planning and policies, also to indicate problems of the current policymaking and implementing process (Cervero, Reference Cervero2013; Annema et al., Reference Annema, Frenken, Koopmans and Kroesen2017). The limitations of using these data are that it is hard to monitor the changes caused by one particular indicator and it usually does not contain all the indicators needed by assessments, which means that it can only support basic information to data analysts (Mozos-Blanco et al., Reference Mozos-Blanco, Pozo-Menéndez, Arce-Ruiz and Baucells-Aletà2018).
4.3. GIS data
GIS is a broadly used information technology that has transformed the ways investigators conduct research and has had tremendous effects on research techniques (Foote and Lynch, Reference Foote and Lynch1996). ArcGIS is one of the GIS applications using technologies that could help geographers to gain multiple categories of spatial data by working with maps and geographic information, which can also be used to compare the data in different timeframes, and to analyze mapped information applied in a wide range of research domains (Johnston et al., Reference Johnston, Ver Hoef, Krivoruchko and Lucas2001). The main feature of GIS data is that it can provide visual displays for data analysts, especially for policymakers as it helps to transfer complicated data in a straightforward and understandable way (Scott and Janikas, Reference Scott and Janikas2010).
According to these characteristics of GIS data, it has been used in urban planning (Maantay and Ziegler, Reference Maantay and Ziegler2006), resource management (Pettit et al., Reference Pettit, Cartwright, Bishop, Lowell, Pullar and Duncan2008), public health (Hirschi et al., Reference Hirschi, Schenkel and Widmer2002), transportations (Thill, Reference Thill2000), and also many other different fields. Researchers mainly applied it to acquire the information of landscapes, streets, public transport lines, and roads lines, which all of them are very useful for mobility studies (Greene and Pick, Reference Greene and Pick2012). Increasingly urban mobility researchers combine GIS data together with the other types of data (i.e., GPS data, mobile phone data, and social media data) to detect urban travel modes and behaviors, which could compensate the limited information provided by GIS data (Gong et al., Reference Gong, Chen, Bialostozky and Lawson2012; Khan et al., Reference Khan, Maoh, Lee and Anderson2016).
4.4. Big data
Big data refers to data in large volumes, is heterogeneous, and has autonomous sources in decentralized control according to the techniques used to explore the complex relationships among the data (Wu et al., Reference Wu, Zhu, Wu and Ding2014). It has the potential to depict overall macrotrends with huge amounts of available, and with a high level of detail, information, which also helps to change traditional ways of collecting and analyzing data in practice and research (Pucci and Vecchio, Reference Pucci and Vecchio2019). Global Navigation Satellite Systems, location-based services, public transportation cards, and so on all generate numerous data as a by-product in these operations (Semanjski et al., Reference Semanjski, Bellens, Gautama and Witlox2016). Big data has been first used in business-oriented domains as the data could measure customers’ performance in which providing rich information and knowledge about consumers’ behaviors and preferences for companies to help them making commercial strategies (Linden et al., Reference Linden, Smith and York2003; Hasan et al., Reference Hasan, Morris and Probets2009). From there, it has gradually spread into other fields. It is a new opportunity for experts to exhaustively grasp people’s mobility behavior in order to implement corresponding policies by analyzing these data from multiple sources (Milne and Watling, Reference Milne and Watling2019). Furthermore, it has already made big contributions to solve urban mobility-related issues, such as real-time traffic monitoring, traffic congestion regulation, and traffic accident management (Abdulazim et al., Reference Abdulazim, Abdelgawad, Habib and Abdulhai2013; Calabrese et al., Reference Calabrese, Diao, Di Lorenzo, Ferreira and Ratti2013; Tamblay et al., Reference Tamblay, Galilea, Iglesias, Raveau and Muñoz2016). The typical application of big data in urban mobility studies employ GPS, smart cards, mobile phones, and social media, which we discuss in a bit more detail.
Since the 2000s, it has been prevalent to collect GPS data from GPS loggers, GPS-phones, and GPS-enabled PDAs. With the size and weight of GPS devices becoming smaller and lighter, new potential for gathering people’s mobility information arose (Stopher et al., Reference Stopher, FitzGerald and Zhang2008; Zheng-chang Reference Zheng-chang2008). GPS data include locations, time, speed, and moving tracks (Stopher et al., Reference Stopher, FitzGerald and Zhang2008). Therefore, more and more projects intend to detect people’s travel behavior by analyzing individual movement from GPS data, especially when the cost of these devices has gradually decreased (Liao et al. Reference Liao, Patterson, Fox and Kautz2006, Liao et al. Reference Liao, Fox and Kautz2007). In urban mobility studies, understanding transportation modes, improving traffic regulation, and evaluating management strategies of road networks are the most commonly applied GPS data fields (Mintsis et al., Reference Mintsis, Basbas, Papaioannou, Taxiltaris and Tziavos2004; Bastani et al., Reference Bastani, Huang, Xie and Powell2011). However, the raw GPS data are usually analyzed directly, without understanding trip purposes or other related contexts (Gong et al., Reference Gong, Morikawa, Yamamoto and Sato2014).
Smart card data have been predominantly used by public transport systems around the world since the automated data collecting system emerged in the last few decades, which offers sufficient data to investigate travelers’ mobility behaviors for transport planning, traffic management, and mobility policymaking (Pelletier et al., Reference Paddeu, Fancello and Fadda2011). Two main characteristics of smart cards are that it is quite convenient to take and durable to use (Lu, Reference Lu2007), which makes it easier to acquire data from smart cards, while it also improves the quality of gathered data compared with magnetic cards. Privacy issue is the biggest concern for card users who do not want to share all of their personal data for analysis (Bagchi and White, Reference Bagchi and White2005).
Mobile phones are becoming an important medium for data analysts to acquire large-scale sensing data used in various domains.Urban spatial planning and management (Louail et al., Reference Louail, Lenormand, Ros, Picornell, Herranz, Frias-Martinez and Barthelemy2014; Pei et al., Reference Pei, Sobolevsky, Ratti, Shaw, Li and Zhou2014) as well as social networks (Phithakkitnukoon et al., Reference Phithakkitnukoon, Smoreda and Olivier2012; Jiang et al., Reference Jiang, Ferreira and Gonzalez2017) are two of the most common areas of study applying mobile phone data, which give fundamental knowledge and experience to other research fields. In terms of urban mobility studies, it not only serves new opportunities and perspectives for investigators to understand people’s mobility behavior by a lower cost approach with large sample size and frequently update datasets, but also supports policymakers to monitor the emerging mobility issues and respond correspondingly through measurements promptly (Calabrese et al., Reference Calabrese, Diao, Di Lorenzo, Ferreira and Ratti2013). Meanwhile, analyzing raw mobile phone data is complex work that needs sufficient knowledge of modeling and computer science that are the basic requirement for data analysts to process the huge amount of data and to detect valuable information (Rojas et al., Reference Rojas, Sadeghvaziri and Jin2016).
Facebook, Twitter, Instagram, Weibo, and so on are the most popular social media platforms for everyone to create their own accounts and share their personal data to others. This type of data has been predominantly used in business analytics in the last decade, for example, companies analyzing social media data to explore what are the most trends, and so on for their business (Kaplan and Haenlein, Reference Kaplan and Haenlein2010). In the urban mobility domain, social media data help policymakers detect driving forces of people’s movement behavior, which could be regarded as convincing evidence to make some changes of the current implemented policies according to travelers’ real needs (Hasan et al., Reference Hasan, Zhan and Ukkusuri2013). Although it can provide more in-depth data for experts compared with the other types of big data that we mentioned before, there is not a uniform format for social media data analysis, which means more attention is needed for classification of it (Grant-Muller et al., Reference Grant-Muller, Gal-Tzur, Minkov, Nocera, Kuflik and Shoor2014). Moreover, the privacy issue should always be taken into consideration when such data are collected and used.
5. Case Studies Analyses
Figure 2 shows the main data type used in each study. There are 32 cases for survey data, 10 for statistical data, 9 for GIS data, and 23 for big data. Twenty of these 74 cases combined at least two different types of data in their studies. Survey data are the most popular data type for combined data use, which has been applied in 14 of these 20 cases. GIS data and big data are also very commonly applied with other types of data in urban mobility policy-related research, 10 and 9 out of these 20 cases, respectively.
The number of cases for each type of data.
Regarding the publication date of these articles, it could be seen from Figure 3 that the research about data use in urban mobility policymaking becomes more popular after 2011, especially for survey and big data. Although big data use in mobility policymaking studies shows a rapid increase after 2015, survey data still play the main role in this research domain.
Publication year of the final reviewed literatures.
5.1. Cases analyses
Each case’s main characteristics and core information, including data types, sources, subjects, regions, policy associated process (according to the policy cycle explained by Howlett et al., Reference Howlett, Ramesh and Perl2009, see Figure 4), and how data used in the cases are summarized in detail in Appendix A. Specifically, what types of data used in each case associated with different processes in a policy cycle are illustrated in Table 2 (please check the serial number of the articles in Appendix A). Furthermore, different types of these studies, including pure academic research and policy practice, are distinguished and shown in the same table as well, which shows that little research has been applied in a real policymaking process.
Policymaking cycle (Howlett et al., Reference Howlett, Ramesh and Perl2009).
These cases found and analyzed are mostly academic studies published in scholar journals studying advancements in assessing policies. Some of them did policy assessments first and then took sustainability into account in the discussion, some of them focused on data use techniques for policy assessments but hardly in practice. There was only one SA cases (De Oliveira Cavalcanti et al., Reference De Oliveira Cavalcanti, Limont, Dziedzic and Fernandes2017) from actual policy practice among all of these 74 cases, which evaluated the sustainability of five urban mobility projects in the Curitiba metropolitan region.
As Table 2 shows, big data is becoming an important resource for urban mobility policy-related studies. Comparably, survey data still play an essential role in the same domain. By analyzing these case studies, we found the strength and limitations of different types of data used in the urban mobility policy-related studies, as follows.
Valuable information and deep insights from different perspectives could be provided by survey data, especially if the respondents are experts in the urban mobility fields. For instance, Mansourianfar and Haghshenas (Reference Mansourianfar and Haghshenas2018) analyzed interview transcriptions with local mobility policymakers and combined this with analysis of governmental documents in an ex ante assessment of policy measures at the neighborhood level, which provided targeted and insightful recommendations for urban mobility policymaking. On the other hand, small sample size and information being out-of-date are two common limitations of survey data applied in these studies, which had been shown evidently in Hirschi et al. (Reference Hirschi, Schenkel and Widmer2002)’s and McGuckin et al. (Reference McGuckin, Zmud and Nakamoto2005)’s studies.
All of the cases applying statistical data in their studies reflect that it is the most convenient way to collect historical mobility data through various document sources. Moreover, it also plays a vital role in comparing the same mobility policy measure implemented in different cities, such as Mozos-Blanco’s (Mozos-Blanco et al., Reference Mozos-Blanco, Pozo-Menéndez, Arce-Ruiz and Baucells-Aletà2018), which compares the sustainable urban mobility plans of 38 Spanish cities by analyzing the relevant documents. The same historical recorded data could be easily acquired through statistical year books and governmental documents among cities, which provides a stable source of data for policy assessments. One common limitation showed in these cases is that the resolution of statistical data is relatively low, which may cause information loss for the assessments. Wiersma et al. (Reference Wiersma, Bertolini and Straatemeier2016), for instance, note that limitations in the statistical data available prevented them from taking social factors well into consideration in their study.
The biggest strength of GIS data is that it can provide adequate geographical transport information, including traffic lines, locations of transport infrastructures, and urban road networks both in national and regional scales. All the cases which applied GIS data as the main data source in their studies mention that various online GIS databases could be found to support their studies, whereas sufficient experience of relevant software use is required to process data and build models.
The prominent advantage of big data application in urban mobility policy studies is that it provides massive information that can give a comprehensive assessment of urban mobility policy measures based on traveler behavior analysis. For example, massive traffic data were applied in Paffumi et al. (Reference Paffumi, De Gennaro, Martini and Scholz2015) and Zeitler et al. (Reference Zeitler, Buys, Aird and Miller2012) for the ex ante assessments of policy options in a decision-making process. Moreover, big data also shows strong adaptability of use in different urban mobility policy domains combined with other types of data, especially with survey data in developing decision-making support tools. This can be seen in Jiang et al.’s (Reference Jiang, Ferreira and Gonzalez2017) and Wismans et al.’ (Reference Wismans, Friso, Rijsdijk, de Graaf and Keij2018) studies. On the other hand, a limitation of employing big data in urban mobility policymaking is that it is difficult to structure the input data sourced or constructing models, which means it costs much more time to process and analyze these data. Jiang et al. (Reference Jiang, Ferreira and Gonzalez2017) specifically mentions that, in practice, it will be challenging for urban mobility policymakers to have enough capacity to process and analyze big data.
5.2. Policy-related analysis
Regarding the data use in the policymaking cycle, Figure 5 shows that according to these cases, data use in urban mobility policymaking cycle mainly focused on policy evaluation, decision-making, and policy formulation phases. Big data is the only type of data that has been applied in all processes of the policy cycle and it has been mostly employed in decision-making processes, focusing on decision-making tools development. For instance, Jiang et al.’s examined how to analyze raw mobile phone data combined with census data and geographical datasets in different models in order to see which model is more effective in translating and gaining information for sustainable urban mobility planning. In Andrenacci and Genovese’s (Reference Andrenacci and Genovese2019) research, floating car data contains information on the travel speed, time, and routes which were continuously detected by devices on board the cars, which helped to obtain information on the journey to be examined in models determining the best policy option. Additionally, big data has also been widely used in policy evaluation, mainly focus on applying real-time traffic data to assess the impacts caused by implemented mobility policy measures in order to make prompt regulations. There are only two cases where data were employed in the implementation process, one of which is from big data. Maranzano et al. (Reference Maranzano, Fassò, Pelagatti and Mudelsee2020) applied traffic data combined with GIS data to assess the early-stage impact of an extended limited traffic zone based on a developed traffic model, which provides in time insights for policymakers to adjust the policy according to the evaluation.
Different types of data use in policymaking cycle.
GIS data are another type of data which have been applied in implementation step to explore the optimized regulation methods for efficient mobility regulation improvement (Wang et al., Reference Wang, Wei, He, Gong and Wang2014). Notably, it has been relatively equally used in policy evaluation, decision-making, and policy formulation phases as well, especially by combined with other types of data. It provides the basic information of road networks, regional maps, and other relevant traffic information for traffic model and decision-making tool development.
Statistical data are a valuable resource for both ex ante policy assessment in policy formulation and ex post assessment in policy evaluation, which could be easily acquired in different cities and regions and also easily be compared based on the same dataset. There is only one policy practice case among the 74 cases, which exacted the information from various governmental and academic documents to evaluate the sustainability of the Curitiba metropolitan region mobility projects (De Oliveira Cavalcanti et al., Reference De Oliveira Cavalcanti, Limont, Dziedzic and Fernandes2017). It provided the policymakers clear sustainability goals to achieve the evaluated mobility projects.
Survey data are the dominant data type used in almost all steps of the policy cycle except implementation and it is one of only two data types that has been embedded in agenda setting processes. Large-scale commuter travel surveys had been used in this step to detect mobility problems and then to set up corresponding policy measures, for instance, McGuckin et al. (Reference McGuckin, Zmud and Nakamoto2005) did a national survey to investigate participants’ daily travel information so as to define mobility problems for policy measure design. Survey data are also the main resources for ex post assessments of urban mobility policies—nearly 53% of all the cases in policy evaluation process employed survey data as the main database, which shows that big data has not replaced this traditional data type in urban mobility policy assessments.
6. Discussion
6.1. Comparisons of strengths and weaknesses of four different data types
Comparing the 74 cases, we can see that the most detailed information obtained for transport policy planning and assessment is from survey data. It not only contains each respondents’ personal information, but also tells of the motivations behind their travel behaviors directly, which is rather useful for developing sustainable policies. However, limited available data, related to the rather time-consuming nature of organizing surveys, is the main weakness of its application. One suitable way to solve this problem is to set a certain target group of responders, for example, 168 respondents in Soria’s (Soria-Lara et al., Reference Soria-Lara, Bertolini and te Brömmelstroet2015) research are EIA developers, transport planners, and some other professionals with transport planning or evaluating experience, providing sufficient valuable information to evaluate the EIA process for urban transportation planning in Spain. Besides this, web-based survey approaches can help to improve efficiency of the data collection by sending easy links to questionnaires to targeted groups.
The application of statistical data in mobility assessment studies is widely practiced as well, especially on a national scale. The main purposes of four (De Grange et al., Reference De Grange, Troncoso and González2012; Haghshenas et al., Reference Haghshenas, Vaziri and Gholamialam2015; De Oliveira Cavalcanti et al., Reference De Oliveira Cavalcanti, Limont, Dziedzic and Fernandes2017; Mozos-Blanco et al., Reference Mozos-Blanco, Pozo-Menéndez, Arce-Ruiz and Baucells-Aletà2018) of the 10 cases which applied statistical data were to establish assessment criteria and to make comparisons among different projects and policies. Another case (Wiersma et al., Reference Wiersma, Bertolini and Straatemeier2016) sheds light on combining statistical data and GIS data together to examine the driving force of car dependency in the Netherlands. The mixed use of data in this research has been analyzed in a spatial context, which provides sufficient knowledge for policymakers to study car dependency caused by different related factors as well as make it easier to see the variety of results among the cities in the Netherlands. Although a large amount of data used in this research aims to solve the research question—how does the spatial context shape conditions for car dependency, social factors, for instance education level, may cause people choose different ways to various destinations, which has not been taken into consideration because of a lack of data.
According to the case analyses, GIS data have an outstanding capacity to do ex ante assessments for mobility policy decision-making compared with other data types, because a variety of policy alternatives can be tested in models to see which one will have the best performance according to different key factors. Financial issues regarding urban transportation planning can be examined by together analyzing GIS and statistical data which was detected through constructing a new methodological approach to measure the spillover effects of transport infrastructure investments in a spatial context dealt with in ArcGIS software (Gutiérrez et al., Reference Gutiérrez, Condeço-Melhorado and Martín2010).It shows that GIS data have somewhat different function in urban mobility research, mostly related to the opportunity to explain the results with maps to policymakers.
Big data has been widely applied together with survey data, road networks data, and GIS data in ex ante assessment and decision-making tool development for urban mobility policy studies. The combination of survey data and GPS data in Zeitler et al.’s project (Zeitler et al., Reference Zeitler, Buys, Aird and Miller2012) for identifying suburban environmental impacts and evaluating mobility policy options is instrumental to get insights into both travelers’ basic needs and motivations, as well as their actual travel behaviors, which help policymakers see the real requirements of commuters. One prominent characteristic of big data is that it can provide massive individual traveling information offered by tracking devices. These data can be used to depict selected groups of travelers’ activities and to assess efficiency of the relevant decision-making. Nevertheless, only relying on big data, especially GPS and traffic data, will cause data sparsity problems, as noted in Zhan et al.’s (Reference Zhan, Zheng, Yi and Ukkusuri2016) research. Not only are mobility researchers trying to explore the potential use of big data in sustainable transportation policies and governance development, but also data mining and analyzing scientists have begun to detect the valuable messages from it, extending the implementation fields of big data. A study (De Gennaro et al., Reference De Gennaro, Paffumi and Martini2016) published on “Big Data Research” has developed five models based on the information provided by GPS and GIS data, aiming to better use data in urban mobility policy evaluation and governance. One issue observed through reflections on the big data cases is that big data use in the urban mobility policymaking process is still mainly supply-driven and hardly demand-driven.
Currently, there are some new opportunities for researchers and policymakers to develop better mobility policies since a new data type, social media data, has been used in the mobility policy assessment process. For instance, 1.5 million social media data elements from Weibo (the biggest Chinese microblogging platform) and 8 million smart card data units have been analyzed in Yang et al.’s (Reference Yang, Heppenstall, Turner and Comber2019) study to explore connections between social activities and mobility behaviors. This created insight in various spatial and temporal trends of urban transport. The study suggests that social media data can also reflect travel motivations from those data sharers, while taking less time to collect (than surveys), because it can be collected online. However, one common challenge for big data analysis in urban mobility studies is data processing. It is difficult to structure and format input data that are from various sources. Nevertheless, modeling and programming are both necessary skills that are required for analysts to deal with big data. Moreover, according to the features of transportation policymaking, real-time data monitoring and analyzing are both significant factors to have an effective assessment for urban mobility policies.
6.2. Potential better use of data in SA of urban mobility policies
Sustainable assessment of mobility policies should give insight into the impact of policies in terms of accessibility, environmental, and social indicators (Black et al., Reference Black, Paez and Suthanaya2002; Costa, Reference Costa2008). Ideally, SAs show possibilities to stimulate transport modal shifts, to reduce private car use, to cultivate people’s green traveling consciousness, and to improve efficiency in urban transport systems (Banister, Reference Banister2008). Data are an essential ingredient in these assessments. The case studies reviewed helped to learn the current use of different types of data in urban mobility policy assessments, of which most of them are academic studies. Furthermore, it also helps to explore the potential of available data innovations of applications for policy practice.
Da Silva et al. (Reference da Silva, de Azevedo Filho, Macêdo, Sorratini, da Silva, Lima and Pinheiro2015) emphasize that data availability and quality are the most important elements to run an assessment, which also depends on whether policymakers and researchers are involved in formulating assessment criteria based on their problem perceptions. Data reliability should be a concern when analysts are going to deal with collected data as it determines problem solving and corresponding measures designing directions (Witlox, Reference Witlox2007). It is also necessary to weight the representativeness of data before using analyzed results into policy assessment, which has been highlighted for transport studies since 1993 (Schoonees and Theron, Reference Schoonees and Theron1993). Moreover, privacy is the most common issue that we must care about when we use individuals’ information for policymaking (Hwang et al., Reference Hwang, Wei and Lee2009; Kifer and Machanavajjhala, Reference Kifer and Machanavajjhala2011; Callegati et al., Reference Callegati, Campi, Melis, Prandini and Zevenbergen2015). A thoughtful way to deal with it is giving announcements to respondents who share their private data for a certain use as well as informing them on the final research results after assessments. Lastly, a practical issue has been mentioned by an EU mobility policymaker in 2019 EU Conference on Modelling for Policy support which was that most of the data currently available for urban mobility policymaking is supply-driven but not demand-driven, which causes policymakers to have limited space when choosing the data they really need. This requires more cooperation among different parties working together to give more opportunities for mobility policymakers gathering the data they need for policymaking.
In order to advance mobility policy assessments in terms of data use, exploring the role of data in various phases of the policymaking cycle and detecting what kinds of skills and expertise are needed for policymakers could be helpful. Firstly, in the agenda setting phase, historical data collection and processing can be used to define and frame problems (Doern and Phidd, Reference Doern and Phidd1983). However, if the statistical data, such as the number of electric vehicles, charging stations, and PM 2.5 emissions, could not be measured periodically in this stage, it would be demanding work for policymakers to define actual mobility problems. This step also requires policymakers to select basic indicators that are easy to collect periodically which is essential for urban mobility issue defining. In the second phase, policy formulation, policy options should be developed and preliminarily ranked. Traditional survey data, GIS data, and big data all show their usefulness for policy option formulation, especially the combined use of survey data and big data has been found to have a big potential to help understand traveler behaviors and corresponding motivations. This will help policymakers design more humane and sustainable transport policy measures. Besides this, ex ante assessment can also be very useful in both this stage and next in the decision-making process, which helps policymakers select the most suitable solutions.
In the third phase, the final policy measure for implementation needs to be decided. Gathering GIS data and traffic data processed in ArcGIS software is an effective way to evaluate and compare different policy options. This gives governments more chances to see different forecast results based on varied scenarios and further to draw a bigger picture of their transportation planning. Analyzing the data in this step requires professional employees such as modelers and data analysts since sufficient data processing and modeling knowledge are needed to dig information from raw GIS and big data. Otherwise, it will cause a common problem facing policymakers where they have a lot of data but they do not know how to select and use it.
In the fourth step, selected policies should be implemented, and in the last phase, monitoring and evaluation of the policies should be conducted. In practice, these two steps are often not sequential but iterative (Hessing and Summerville, Reference Hessing and Summerville2014). Big data, such as real-time traffic data, GPS data, mobile phone data, and social media data, can give more in-time reflections of implemented policies in this period, which could let policymakers make prompt adjustments responding to the problems showed in the policy implementation phase. Ex post evaluation can also employ statistical data and survey data to compare outcomes of current policy with those of previous policies as well as to analyze feedback from travelers after policy trails, which is an important step to respond to potential problems. However, this also requires sufficient work capacity from the urban mobility departments to conduct monitoring and evaluations.
7. Conclusion
In this paper, we review recent (2000–2020) academic literature on urban mobility policy assessments to understand the current state-of-the-art of data use in these activities and further explore the potential of available data innovation in more evidence-based policymaking. The 74 case studies reveal a surge of attention and availability of open, big data, although, it cannot replace traditional data usage (surveys, statistics). We do find that the new types of data provide new opportunities for evidence-based policymaking.
Overall, the data use innovations in SA for urban mobility policy can be concluded as follows: (a) big data shows the most potential for use in decision-making support tools development, especially combined with survey data which shows even higher effectiveness; (b) Specifically, big data (most of the available big data are location-based data) used in traffic models can more easily provide detailed information about travel patterns, but reveals less about motivation while traditional surveys remain more useful for this; and (c) The use of new types of data in urban mobility policymaking requires policymakers and related working staff to have certain knowledge and skills for data analysis, modelling and extra working capacities.
7.1. Limitations and future research
In the literature search and selection process, it was a criterion that studies shed light on the use of data in urban mobility policies, especially for policy assessments toward sustainability. Because of this real-place particular focus, a broad range of mobility policy assessments in the literature are left out. Additionally, only one of the case studies is based on policy practice, while the others are all academic research, so we can hardly conclude with extremely certain suggestions for urban mobility policymakers in practices.
Moreover, because the 74 cases are mostly academic studies, not from actual policy practice, it is a gap that should be addressed by future research since innovation in policy assessment likely takes place in practice as well. This can lead to better understanding of the use of state-of-art of data in practice and recommend the most optimal use of new data types used in urban mobility policymaking. The studies we reviewed did not reveal how policymakers appreciate the various data types and how they are involved in shaping data analysis. It seems like there is a tendency for supply-driven data in practice as well. Studies of innovation in policy assessments in practice can reveal the best applications, constraints, and potential of more demand-driven data use in mobility policy assessments.
Acknowledgments
The author is grateful for the support provided by our interviewees and the insightful comments from Joop de Kraker.
Funding Statement
This research was supported by grants from the China Scholarship Council. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests
The author declares no competing interests exist.
Author Contributions
Conceptualization: M.D. and X.L.; Methodology: X.L. and M.D.; Data curation: X.L.; Data visualization: X.L.; Writing original draft: X.L. and M.D. All authors approved the final submitted draft.
Data Availability Statement
Data availability is not applicable to this article as no new data were created or analysed in this study.
Appendix A
Analysis highlights of the reviewed literature
Open access
Comments
No Comments have been published for this article.