2.1. Partial digital governmental transformation

ICT-enabled organizational reforms of the public sector have had a long history in the UK (as elsewhere), closely connected to New Public Management goals of efficiency, target delivery, outsourcing, and competition (Dunleavy et al., Reference Dunleavy, Margetts, Bastow and Tinkler2006; Tassabehji et al., Reference Tassabehji, Hackney and Popovic2016). These reforms have been discussed using several overlapping frameworks, including “e-government,” “digital government,” “digital era governance,” and more recently, “transformational government” and “digital government transformation” (Dunleavy et al., Reference Dunleavy, Margetts, Bastow and Tinkler2006; King and Cotterill, Reference King and Cotterill2007; Meijer and Bekkers, Reference Meijer and Bekkers2015; Vial, Reference Vial2019; Liva et al., Reference Liva, Codagnone, Misuraca, Gineikyte and Barcevicius2020; Tangi et al., Reference Tangi, Janssen, Benedetti and Noci2021). Notably, ICT-enabled organizational change has been discussed from a sociotechnical perspective that understands organizations as complex sociotechnical systems (see Sawyer and Jarrahi, Reference Sawyer, Jarrahi, Topi and Tucker2014). The technical system comprises “processes, tasks, and technologies needed to transform input into output” (Nograsek and Vintar, Reference Nograsek and Vintar2014, p. 110), while the social system encompasses people, structure, and culture (Nograsek and Vintar, Reference Nograsek and Vintar2014, p. 110). For organizations to change, transformation must occur within both systems (Nograsek and Vintar, Reference Nograsek and Vintar2014; Weerakkody et al., Reference Weerakkody, Omar, El-Haddadeh and Al-Busaidy2016; Faro et al., Reference Faro, Abedin and Cetindamar2021; Tangi et al., Reference Tangi, Janssen, Benedetti and Noci2021). For example, in their empirical analysis of Italian public sector organizations, Tangi et al. (Reference Tangi, Janssen, Benedetti and Noci2021) showed that, while digital technologies enabled the transformation of the technical system, the corresponding social system was left largely unaffected. Digital transformation, thus, needs to go beyond the mere digitalization of services and processes to include structural and cultural organizational changes (Weerakkody et al., Reference Weerakkody, Omar, El-Haddadeh and Al-Busaidy2016; Mergel et al., Reference Mergel, Edelmann and Haug2019; Tangi et al., Reference Tangi, Janssen, Benedetti and Noci2021). Following this literature, this article understands local government’s data engagement during the pandemic as embedded in broader processes of digital transformation (whether successful or not) at both technical and social system levels.

The literature identifies several challenges and barriers to public sector digital transformation, which broadly map onto the social–technical perspective of organizations. Technical barriers include the complexity and incompatibility of technologies, security threats, lack of and outsourcing of infrastructural facilities, lack of technical standards and interoperability, and system legacies. Barriers related to organizational structures encompass lack of managerial and/or political support, lack of skills and knowledge, lack of available resources (e.g., funding and staff), organizational complexity and fragmentation, while cultural barriers include resistance to change, bureaucratic culture, and risk aversion (Gil-García and Pardo, Reference Gil-García and Pardo2005; van Veenstra et al., Reference van Veenstra, Klievink and Janssen2011; Meijer, Reference Meijer2015; Barcevičius et al., Reference Barcevičius, Cibaitė, Codagnone, Gineikytė, Klimavičiūtė, Liva, Matulevič, Misuraca and Vanini2019; Wilson and Mergel, Reference Wilson and Mergel2022). The literature, furthermore, points to external factors that can drive, or impede, innovation and change, including policies and regulations, collaborations and engagement in interorganizational networks, and pressure from external stakeholders (Cinar et al., Reference Cinar, Trott and Simms2019; Tangi et al., Reference Tangi, Janssen, Benedetti, Noci, Viale Pereira, Janssen, Lee, Lindgren, Rodriguez Bolivar, Jochen Scholl and Zuiderwijk2020). Crises, as part of the external environment in which organizations operate, has been identified as a potential driver of digital government transformation (Pittaway and Montazemi, Reference Pittaway and Montazemi2020; Tangi et al., Reference Tangi, Janssen, Benedetti and Noci2021; Eom and Lee, Reference Eom and Lee2022), as the following section elaborates.

2.2. Mixed evidence of the impacts of crises on public organizations

Crises are characterized by three key components: threat, uncertainty, and urgency (Boin et al., Reference Boin, Hart, Kuipers, Rodríguez, Quarantelli and Dynes2007, p. 43). Importantly, they need to be understood as extended periods and not just limited to a triggering event (Boin et al., Reference Boin, Hart, Kuipers, Rodríguez, Quarantelli and Dynes2007; Williams et al., Reference Williams, Gruber, Sutcliffe, Shepherd and Zhao2017). The literature provides mixed evidence of the impacts of crises on organizational change. On one hand, crises are often viewed as external shocks that generate “windows of opportunity” for organizational reforms as well as wider policy change (Boin et al., Reference Boin, Hart, Kuipers, Rodríguez, Quarantelli and Dynes2007). Accordingly, as disruptions to normal routines and practices, crises demand novel and rapid responses which, in turn, can foster innovation (Bessant et al., Reference Bessant, Rush and Trifilova2012; Avelino et al., Reference Avelino, Wittmayer, Haxeltine, Kemp, O’Riordan, Weaver, Loorbach and Rotmans2014; Gkeredakis et al., Reference Gkeredakis, Lifshitz-Assaf and Barrett2021; Oborn et al., Reference Oborn, Pilosof, Hinings and Zimlichman2021). Orlikowski and Scott (Reference Orlikowski and Scott2021), for example, argue that crises create opportunities to engage in “liminal innovation,” by leveraging the tensions arising from disruptions to improve established organizational practices and processes. This may include the use of iterative experiments, open-ended sense-making and alternative ways of doing things (Orlikowski and Scott (Reference Orlikowski and Scott2021)). Drawing on this literature, emerging research documents how the COVID-19 pandemic has reconfigured ways of working and generated the conditions for innovation, experimentation, and collaborations (Mazzucato and Kattel, Reference Mazzucato and Kattel2020; Gkeredakis et al., Reference Gkeredakis, Lifshitz-Assaf and Barrett2021; Orlikowski and Scott, Reference Orlikowski and Scott2021).

On the other hand, the notion of crisis as window of opportunity has been criticized with the literature pointing to the complex dynamics at play in relation to the transformative potential of crises in public administrations (Boin et al., Reference Boin, Hart, Kuipers, Rodríguez, Quarantelli and Dynes2007, Reference Boin, Hart and McConnell2009). The organizational capacities to transform and adapt in response to crises depend on factors such as leadership, communication, and legitimacy, as well as on preexisting organizational arrangements and culture (Deverell and Olsson, Reference Deverell and Olsson2010; Boin et al., Reference Boin, Stern and Sundelius2016; Christensen et al., Reference Christensen, Lægreid and Rykkja2016; Tokakis et al., Reference Tokakis, Polychroniou and Boustras2019). Furthermore, crises often generate “political framing contests” in which various actors seek to “interpret events and the responsibilities and lessons involved in ways that suit their political purposes and visions of future policy directions” (Boin et al., Reference Boin, Hart and McConnell2009, p. 82). These discursive framings can create competing pressures for continuity and change (Boin et al., Reference Boin, Hart and McConnell2009). Thus, whether crises are “triggers of systemic change, or if they serve to forestall such change” remains an “open question” (Boin et al., Reference Boin, Hart, Kuipers, Rodríguez, Quarantelli and Dynes2007, p. 52). This debate aligns with the question of incremental versus radical change in the wider public policy and organizational change literature (Kuipers et al., Reference Kuipers, Higgs, Kickert, Tummers, Grandia and Van der Voet2014; Van der Heiljman and Kuhlmann, Reference van der Heiljman and Kuhlmann2017).

Emerging research findings on the impacts of the COVID-19 pandemic point to accelerated digitalization in public sector organizations, particularly related to the needs to rapidly provide public services digitally and to the move to remote working (Agostino et al., Reference Agostino, Arnaboldi and Lema2021; Aristovnik et al., Reference Aristovnik, Kovč, Murko, Ravšelj, Umek, Bohatá, Hirsch, Schäfer and Tomaževič2021). At the same time, the effects the crisis has had on, and the challenges it poses to, broader organizational processes associated with digital transformation are not yet clear (Gabryelczyk, Reference Gabryelczyk2020; Eom and Lee, Reference Eom and Lee2022). In response, this article examines local governments’ data engagement in the face of the crisis and analyses whether this led to organizational changes in both technical and social systems (see research questions, Section 1). In doing so, the article contributes to the analysis of the dynamics (enablers and barriers) of digital government transformation, and the potential role of crises as enabler of systemic, sociotechnical change in public sector organizations.

3.1. A case study of Scottish local government

This research focuses on local government since it was on the frontline of efforts to combat the COVID-19 pandemic and experienced heightened demands for data intelligence. As such, it lends itself to studying how the crisis affected data engagement in a public sector setting and what, if any, digital transformations emerged from this. Scotland was chosen as case study for three reasons: first, the digital transformation of the public sector has been a key policy of the Scottish government for over a decade (see below). Second, as a relatively small nation (5.5 m inhabitants), Scotland consists of 32 local authorities which, for the purpose of systematic research, is methodologically manageable. Third, in practical terms, an opportunity arose to implement the research with support from the Digital Office for Scottish Local Government (henceforth “Digital Office”), thereby helping to ensure access to local authorities and other public sector organizations and, afterwards, to disseminate the research results and, thus, inform ongoing data practices.

The Scottish government published its original digital strategy “Scotland’s digital future” in 2011 (updated in 2017), followed by the publication of the Open Data Strategy in 2015 (Scot Gov, 2011, 2015). This translated into several initiatives to support digital transformation in local government, most notably the establishment in 2016 of the Scottish Local Government Digital Transformation Partnership involving all Scottish local authorities. From this emerged the Digital Office which provides support to councils in four areas of digital transformation: digital leadership; digital foundations; digital services; and digital telecare. On its part, the Improvement Service, the national improvement organization for Scottish local government, also supports local authorities in their digital transformation, particularly relating to the use of spatial data. Additionally, the Scottish Cities Alliance (SCA), jointly established by the seven Scottish cities and the Scottish Government in 2011 to advance the Agenda for Cities (Scot Gov, 2016), includes in its smart city programme collaborative projects on open data, smart communities, smart services, and smart infrastructures. Early 2021, during the pandemic period, the Scottish government published its latest digital strategy, “A changing nation: how Scotland will thrive in a digital world” (Scot Gov, 2021). The strategy was jointly developed with the Convention of Scottish Local Authorities (COSLA), the Digital Office, and the Improvement Service, thus underscoring the importance attached to local government in the pursuit of digital transformation in the public sector. The document refers to the pandemic as a key moment that demonstrated the importance of good data, data intelligence and digital transformation (Scot Gov, 2021, pp. 91–92). The strategy puts forward several practice-oriented recommendations, including: building common platforms across the public sector; developing and accelerating the adoption of common digital and data standards; fostering greater collaboration; securing data sharing within and across organizations; and implementing a Data Transformation Framework to ensure data quality and encourage the expansion of data reuse across the public sector (Scot Gov, 2021).

3.2. Mixed method approach

The research was conducted during the height of the pandemic, under a national lockdown. In line with institutional ethical guidance, a decision was taken not to conduct research with frontline staff directly involved in public service delivery, given the added pressures under which they operated. Instead, staff with more strategic or coordinating roles was invited to participate. This had the benefit that they had a good overview and understanding of their typically large and complex local government organizations. Research participants were selected based on their active involvement in data use/management (“data cohort”) and pandemic response (“recovery cohort”). They, thus, provided situated information and perspectives on local government’s data-related responses to the crisis. As such, however, their responses do not represent the full range of experiences and viewpoints from across the multidepartmental local government organizations. Finally, it should be noted that participants were invited to engage in the research in an individual professional capacity. Their responses, therefore, should not be read as official viewpoints of the local authorities concerned.

The research used a mixed method approach starting with a survey of all 32 local authorities and followed by three focus groups and five semistructured interviews. The fieldwork, for which institutional ethical approval was obtained, was conducted between October 2020 and April 2021.The Digital Office provided practical assistance, including recommending the survey to the local authorities and facilitating contacts with potential participants for the focus groups. They also provided policy- and practice-relevant feedback on the draft survey questionnaire and supported the dissemination of results. Responsibility for the research (design, ethics, implementation, and analysis), however, rested solely with the authors.

3.2.1. Survey

The survey was designed based on (a) a review of relevant academic and policy literatures, and (b) advice from data scientists, public policy researchers and Digital Office staff. The survey was piloted with six volunteers and adjusted. The final version contained 20 questions and was structured around six main themes: “General data response to COVID-19”; “Data uses and needs” (including specific data types, sources, policy domains); “Data capabilities” (incl. gaps and opportunities); “Data sharing and collaborations”; “Data opportunities and challenges”; and “The role of data in the context of COVID-19.” The full questionnaire is available in https://researchdata.gla.ac.uk/1147/?_ga=2.12681719.1187732427.1657116747-1022996968.1651148229. The rationale was to include all Scottish local authorities (32) and in each authority capture information from, on one hand, a staff with an experienced role in data management (digital services, business intelligence, innovation, and research) and, on the other, a staff with a central role in COVID-19 recovery planning. The sample, therefore, comprised 64 local government officers: 32 in the “data” cohort, and 32 in the “recovery” cohort (the questionnaire contained questions common to both cohorts, as well as specific to each, see questionnaire). The officers were identified from the database of the Digital Office as local authority personnel involved in the Scottish digital transformation partnership (data cohort) and COVID-19 response (recovery cohort). To safeguard anonymity, the names were withheld from the researchers (the Digital Office sent out the survey invitation on behalf of the researchers). The survey response rate was 70.3%, with 31 local authorities (96.8%) participating (see Table 1). Thus, it was above average of expected response rates in organizational research (Holtom et al., Reference Holtom, Baruch, Aguinis and Ballinger2022).

Table 1.

Survey responses per cohort

3.2.2. Focus groups

Three separate focus groups (FG) were held with local government officers, representatives of public sector organizations, and representatives of third sector organizations (see Table 2). FG1 brought together officers from five local authorities (different sizes, geographic locations, and urban–rural makeup), to elicit qualitative information and in-depth discussion about local authorities’ data needs, capabilities, and uses resulting from COVID-19. Participants were drawn from the survey sample. The other two focus groups were designed to elicit information and discussion about data sharing practices and collaborations between local authorities and public sector organizations (FG2), and between local authorities and the third and voluntary sector (FG3). Participants for FG2 were recruited with the help of the Digital Office; they included, alongside three local government officers, four staff with strategic and/or coordinatory roles in their respective public sector organizations (government, police, health service). Participants for FG3 were recruited through desk research; they included, alongside two local government officers, six staff in voluntary and community organizations that participated in the pandemic response and recovery either by supporting local communities directly (e.g., food community charities) or coordinating voluntary organizations at regional or national levels (e.g., SCVO and SCDC).

Table 2.

Focus groups participants

Each focus group started with an open prompt inviting participants to reflect on their organizations’ overall data response to COVID-19 and ended by asking participants to discuss emergent opportunities for data engagement and digital transformation. This common structure was used to allow for comparison across the three focus groups. Additionally, FG1 included two bespoke discussion prompts, on types of data used to respond to COVID-19, and on challenges related to data use and capabilities. FG2 and FG3 also encompassed two additional prompts; one concerning the accessing and sharing of external public sector data (FG2) and third sector data (FG3), and the other concerning data-related collaborations between local authorities and their respective sector.

3.3. Data analysis

The quantitative data collected from the survey was aggregated, using descriptive statistics, and analyzed to identify key themes. On this basis, the survey findings initially resulted in five main categories:

1. 1. Importance of public sector data;

2. 2. Intensification of data use mainly to deliver essential services;

3. 3. Key challenges of data quality and data harmonization;

4. 4. Significance of third sector data (albeit underutilized)/missed opportunity of private sector data use;

5. 5. Limited use of novel data.

The data from the focus groups and interviews were transcribed and coded in NVivo, using an iterative thematic analysis approach: the coding matrix was first developed based on the initial five thematic categories (see above), following which it was refined based on the summary notes of each focus groups and interviews. This resulted in five additional themes as follows:

1. 1. Interorganizational governance;

2. 2. Intraorganizational complexity and legacy systems;

3. 3. Data ethics, security, and public trust;

4. 4. Opening of conversations and change of perspectives in relation to data;

5. 5. Defining and aligning the purpose of data use/collaborations with existing needs.

Each theme emerged from the clustering of coding nodes across the datasets. For example, theme 2 included the following nodes: “increased data use,” “high pace response,” “accelerated data sharing and collaborations,” “focus on essential services,” “operational needs,” “initial response,” and “collection of new types of data.” Analytical and visualization tools in NVivo and Microsoft Excel (crosstab and frequency queries, bar charts, sunbursts, tree maps, etc.) were used to identify trends and patterns across the quantitative and qualitative datasets. Methodological triangulation (Flick, Reference Flick2007) was applied to ensure the robustness of the research and cross-examine the validity of findings. This included checking emerging themes across datasets, comparing and ensuring consistency of research notes, independently reviewing survey data aggregation and analysis, documenting the development of the coding matrix over time, and presenting methodology and initial analysis of findings to colleagues and stakeholders.

4.1. Crisis response: Intensification of public sector data use and sharing

A key finding of this research is that Scottish local government’s data engagement in the initial phases of the COVID-19 crisis was predominantly centered on public sector data. Survey respondents identified internal and external public sector data as the most important types of data during the first month of the lockdown (89 and 67%, respectively) and in the period October–November 2020 (80 and 76%). Third sector data (e.g., data on food and medicine distribution and numbers of volunteers) was also seen as important, with two thirds of respondents rating it “quite/very important” in the same periods. However, only 44% of survey respondents reported using it, thus highlighting a discrepancy between perceived importance and actual use. Strikingly, respondents rated the importance of public sector data far higher than that of private sector data and novel data (e.g., cellular, crowdsourced, and sensor data). Private sector data was considered by a significant proportion of respondents as either “not at all important/fairly unimportant” (31% concerning the first month of lockdown; 20% concerning the period October–November 2020) or “neither important nor unimportant” (38 and 31%, respectively). In addition, participants reported relatively low usage of private sector data (27%; survey, Q. 11). The findings on novel types of data are similar, with two thirds of respondents rating them as “neither important nor unimportant” or “not at all important/fairly unimportant” in both periods. The reported use of novel types of data was limited, namely in descending order: social media data (24%), data from connected infrastructures (18%), cellular and Wi-Fi data (4%), and crowdsourced data (4%). Significantly, 36% of respondents reported that their local authority had not used any (survey Q. 15).

The prevalence of public sector data (internal and external) is in large parts explained by the public health nature of the crisis and the associated responsibilities of central and local governments and health boards. The survey data confirms that local government experienced an acute need for data concerning essential public services, including adult social care, education, and children services. Data needs also increased in the economic domain, in relation to business and corporate services and emergency support for local businesses (see Table 3). Data needs were made more acute by new data requirements prompted by the Scottish government and COSLA, which requested regular data returns from local government. Local government participants described these requirements as an onerous challenge, especially at the beginning of the crisis.

Table 3.

Ranking of policy areas with increased data needs resulting from COVID-19 (survey Q.7, both cohorts)

The need for rapid data intelligence was met by both an intensification of data use within local government and an increase of data sharing across the public sector. Public sector data used by local government included various operational and administrative data on: the uptake of public services (e.g., adult social care data and customers contacts data); social benefits recipients (e.g., free school meals and housing benefits); local government employees (to facilitate staff redeployment); education (e.g., teachers’ absence and infection outbreaks in schools); and local businesses (e.g., business grant and rate relief data). Local authorities also used external public sector data, notably personal data provided by the NHS to local government on vulnerable people required to shield (self-isolate) and data related to the spread of the pandemic (e.g., local infection and death rates).

Seventy percent of respondents reported an increase in the integration and analysis of existing internal data, and 83% an increase in internal data sharing. This occurred alongside the acquisition of new data, with 78% of respondents reporting using new data sources and 74% indicating an increase in data collection (see Table 4). As one focus group participant (local authority) put it, “our use of data almost from day one really went through the roof.” Local government’s main data sources were other public sector organizations, including NHS Health Boards (93%), the Scottish Government and its agencies (78%), and Special NHS Boards (67%) (Survey Q. 10, both cohorts). Relatedly, 70% of respondents described an increase in data sharing externally and 65% reported the implementation of data sharing agreements in response to COVID-19 (see Table 4).

Table 4.

Local government’s data response to COVID-19 (survey Q. 6, data cohort)

Local government’s response in the initial stages of the crisis was focused on using data to identify and support vulnerable groups, as well as monitoring and adapting service provisions:

Across all datasets, participants spoke of the rapid and collective efforts put in place across organizational boundaries to use and share data to identify and support vulnerable groups. While, however, there was an alignment of purpose, the initial period was marked by a time-consuming case-by-case approach to developing data sharing agreements between local government and NHS health boards concerning shielding data. As one participant recounted:

Furthermore, local governments had to meet new data reporting requirements by the Scottish Government and COSLA, which created additional challenges. As one participant described, “it was new data, it was different data and we did not have the systems to handle this data” (Focus group, local authority). Another explained that the key challenges in the initial response to the crisis were the “rapidity that people required data in, how quickly they needed it” as well as “the transition from manual things” to more systematic ways of organizing the delivery of essential services (Focus group, local authority). To meet these challenges, local government had to develop processes to match and integrate existing datasets with newly acquired shielding data, subsequently expanding to other national datasets:

The integration of different datasets, then, was often accompanied by the development of dashboards to facilitate day-to-day operational decision-making and strategic leadership. Dashboards were used in different areas, for example, to monitor COVID-19 local infections, school absences and educational capacity, local economic and business impacts, and to support the redeployment of staff. The survey data confirms a significant increase in the use of data visualization tools (see Table 4).

Altogether, in response to the first research question (RQ1), these findings confirm the important role data played during the initial stages of the pandemic: local government experienced an intensive demand for data matched by new data reporting requirements, especially related to essential services. Significantly, the increased data engagement was centered mainly on public sector data (internal and external) and to a lesser extent on third sector data, with private and novel data playing only minor roles. This can be explained by the fact that the crisis was centered on health and health-related data. Private sector and novel data were both perceived by respondents to acquire a growing importance in the future (69 and 62% of survey respondents rated private sector and novel data respectively as quite/very important’ for the third period).

4.2. Changes in socio-technical systems: Shortening of information sharing processes and development of new ways of working

Turning to RQ2, the findings point to several organizational changes, in response to increased data demands, at both technical and social system levels within local government, as well as in the interactions with other public sector organizations. Changes were particularly apparent in relation to information systems and agreements to share data within and across organizational boundaries. Participants noted a significant acceleration of processes related to data sharing and governance, but also related to more technical aspects, such as data cleansing and integration:

Another participant recalled how they were now able to “turn something around in a few days that normally would have taken about 4 or 5 months” (Focus group, local authority). The rapid demands for data also led to the adoption of new cross-organizational systems and agreements, including the adoption of a new Data Sharing Framework between thirteen local authorities and NHS NSS, and the development of a Local Government Data Portal to deposit regular COVID-19 data returns to Scottish Government and other regulatory bodies (Digital Office, 2020). Another example of changes to technical systems was the adoption of new, or the repurposing of existing, internal information systems, particularly Customer Relationship Management Systems (CRM):

Other participants reported the deployment of systems and solutions to manage helplines and customers’ contacts, for example Oracle Service and Verint CRM.

The research also sheds light on changes more closely connected to social systems. This included the development and strengthening of cross-organizational networks and collaborations. Participants spoke about an increased openness to collaborate and share data:

Another participant explained that prior to the crisis data sharing used to be seen as something very difficult to achieve, whereas “now it is seen as something that we just work through and get done” (Interview). Collaborating was viewed as a pragmatic and efficient way of rapidly developing complex data sharing agreements and avoiding the duplication of efforts. One participant recalled how stakeholders from different local authorities joined forces to develop a data sharing agreement template that could be shared with others:

Apart from the practical benefits, collaborations were also welcomed more broadly as spaces for peer support and knowledge exchange across the public sector. As another participant explained:

Significantly, these social system-level changes were instrumental in facilitating changes at the technical systems level, thus highlighting the interrelationship between the two systems: the aforementioned Data Sharing Framework (btw. local government and NHS NSS) was led by the COVID-19 Data and Intelligence Network (Scottish government), and the Local Government Data Portal by the Digital Office’s Local Government Data Taskforce.

As to wider organizational culture, some participants also indicated a shift in the recognition of the value of data intelligence for decision-making. As one put it: “suddenly people recognize the value of data and the reason for having data and using it” (Focus group, local authority) while another emphasized that the pandemic had “moved the conversation about data” beyond data practitioners (Focus group, local authority). Another participant saw this shift among senior management and the political leadership: “[spatial data] is becoming really well regarded by the senior people who [had] perhaps thought [that] these geospatial things were, you know, nice to have, but they are now seeing the value of it” (Interview). One focus group highlighted the opportunity for data intelligence and data accessing across the local government organization, exemplified the successful implementation of dashboards and their adoption by the leadership:

Other participants, however, argued that still more was needed to increase the recognition of the value of data as part of the wider organizational culture, especially from people “at the very top” (focus group, local authority). One commented that there was greater need for “a more open approach” to data sharing (Focus group, local authority). Survey respondents also pointed to the need for “greater awareness of the power and importance of data” and for recognition of “how powerful data can be” (Survey Q.19, recovery cohort). Altogether, this suggests that while there were some notable shifts in the organizational culture and social practices occurred, these changes were not evenly distributed and often seen as at risk of backsliding, as discussed further below.

4.3. Sustaining transformation: Addressing persistent challenges and nurturing collaborations

In response to RQ3, the findings provide insights into perceived enablers of, and barriers to, sustaining digital transformation beyond the initial stages of crisis. First, participants pointed out several persistent, preexisting challenges related to data engagement which the crisis had exposed and/or amplified. Inconsistent data quality, and the lack of data standards and common identifiers, were singled out as particularly problematic. Survey respondents rated “ensuring data quality/standards” and “integrating data” between 4 and 5 (on a scale 1–5, from “not at all challenging” to 5 “very challenging”). These two main challenges were followed by “identifying data needs,” “mapping existing internal data,” and “analyzing data” which received slightly lower ratings (see Figure 1).

Figure 1.

Data related challenges experienced by local government (percentage, survey Q. 17, data cohort).

These challenges were especially felt in the initial stages of the crisis when local governments had to match and integrate data from their own organizations with shielding data from NHS health boards. Preexisting gaps in data infrastructures and skills were also felt acutely:

Participants also described significant challenges from multiple legacies of past and existing IT systems as well as from the organizational complexity and siloed nature of local government. Looking ahead, participants, therefore, pointed to the need for appropriate and long-term investment, associated with both the technical and social systems, to be able to address these persistent challenges and support local government in adopting data standards:

Another participant spoke of the needs to invest in data literacy and related skills across local government to ensure that data is used in support of policy and decision-making:

A lack of sustained investment was raised as a major risk of reverting to precrisis practices, or leading to a new, but unsustainable situation where local government would be expected to maintain its data engagement without further resourcing. One participant described this as “trying to make the transformational change while also providing a business-as-usual service with a small team” (Interview). Some participants perceived the crisis as an opportunity to make a strong case for investment, directed at senior leadership. Participants also viewed the pandemic as a catalyst for change in local government’s data engagement and broader digital transformation. One participant described it as “a flash of data maturity modeling” (Focus group, local authority), while another argued that the pandemic had brought 10 years’ worth of digital transformation within local government (Interview). However, as reflected above, other participants were concerned that local government would “retreat back into business as usual after this and keep on doing things as we were” (Interview). The risks of reverting to pre-Covid status quo were seen as particularly acute with respect to data sharing and collaborative working across the public sector:

Participants identified working collaboratively across organizational boundaries as an important factor to maintaining newly gained practices and transformation. Forty-three percent of survey respondents identified collaborating with other stakeholders to collect, use and/or analyze data as a significant opportunity (survey, Q. 18, data cohort), while a focus group participant stated that:

Participants also saw the sharing of learning and good practices as a particular benefit of collaborations which, in turn, would reduce unnecessary duplication of efforts. As one participant put it:

In this context, participants spoke to the importance of strengthening and developing cross and interorganizational “networks of data practitioners” (Focus group, local authority), to share learning and work collectively toward solutions. Finally, local government collaborations were also discussed as a means to gain collective bargaining power and negotiate better data accords and agreements with data owners, in particular private sector organizations, to access and reuse relevant data: