2.1. Policy relevance of CDR data

There is a rich and growing literature seeking to leverage the potential of CDR data to inform policy-making. With mobile penetration rising in developing countries (GSM Association, 2020), researchers have demonstrated the use of CDR data to create poverty maps (Blumenstock et al., Reference Blumenstock, Cadamuro and On2015), understand migration patterns, and estimate a household’s economic characteristics (Blumenstock, Reference Blumenstock2018). There have been sustained improvements in forecasting population density by combining high-resolution satellite data with powerful algorithms (Stevens et al., Reference Stevens, Gaughan, Linard and Tatem2015). WorldPop trains its algorithms on historical census data and projects annual population density at 100-m resolution in a publicly available dataset (Stevens et al., Reference Stevens, Gaughan, Linard and Tatem2015). However, these datasets rely on slow-moving indicators and are computationally intensive. Forward-looking projections are based on linear extrapolation and do not account for short- or medium-term population movement dynamics.

Researchers have harnessed mobile phone data to map population movements. Deville et al. (Reference Deville, Linard, Martin, Gilbert, Stevens, Gaughan, Blondel and Tatem2014) showed that the density of unique users in a cell tower’s catchment area scales with population density, and can be plotted on a logarithmic curve. Researchers can therefore extrapolate shifts in the number of unique users to predict shifts in population densities over time, day-by-day or week-by-week. Accordingly, CDR aggregates provide insights on population movements, which are useful for estimating regional connectivity and the impact of mobility restrictions (Wesolowski et al., Reference Wesolowski, Buckee, Engø-Monsen and Metcalf2016). It also helps to identify areas with higher risks of importation due to population flows from other regions, and develop spatial epidemiological models (Aledort et al., Reference Aledort, Lurie, Wasserman, Bozzette, Lurie-Nicole, Wasserman-Jeffrey and Bozzette-Samuel2007; Wesolowski et al., Reference Wesolowski, Eagle, Tatem, Smith, Noor, Snow and Buckee2012).

Combining CDR data with administrative and survey data offers insights on fast-moving health and well-being indicators. Drawing from survey data on the incidence of poverty in Rwanda, Blumenstock et al. (Reference Blumenstock, Cadamuro and On2015) use machine learning algorithms to predict poverty outcomes based solely on patterns in mobile network data. Zu Erbach-Schoenberg et al. (Reference Zu Erbach-Schoenberg, Alegana, Sorichetta, Linard, Lourenço, Ruktanonchai and Tatem2016) combine CDR data with public health datasets in Namibia to link mobility and malaria incidence. When compared with estimates using static maps, this leads to discrepancies of up to 30%. These applications showcase the value-added of high-resolution, high-frequency proxy data like CDR in the context of an epidemic such as COVID-19.

CDR data can also be used to rapidly update estimates of population distribution when a natural disaster leads to widespread displacement. Bengtsson et al. (Reference Bengtsson, Lu, Thorson, Garfield and von Schreeb2011) used CDR data from a major telecommunications company to track displacement in Haiti after the 2010 earthquake. This allowed them to track shifts in population distribution and estimate that up to 20% of the population of the capital city left in the 19 days after the earthquake. Lu et al. (Reference Lu, Wrathall, Sundsøy, Nadiruzzaman, Wetter, Iqbal and Bengtsson2016) used CDR data to track short-term mobility in the hours and days after Cyclone Mahasen hit Bangladesh in May 2013.

2.2. Methodological innovations CDR data

The most commonly used methods for processing CDR data are traditional data mining techniques. These include frequency-based analysis, data clustering using unsupervised machine learning, and geo-visualization techniques by mapping geolocation (Calabrese et al., Reference Calabrese, Ferrari and Blondel2014). In recent years, researchers have used de-identified CDR data to compute origin–destination (OD) matrices in order to better map patterns in travel behavior (Calabrese et al., Reference Calabrese, Di Lorenzo, Liu and Ratti2011). In combination with additional administrative or survey data, supervised machine-learning methods can inform the prediction of outcomes based on patterns in the cell network (Sundsøy et al., Reference Sundsøy, Johannes, Reme, Iqbal and Jahani2016).

The preprocessing of the raw CDR data is essential to accommodate positioning errors in data collection and the first step for processing. The oscillation problem of the user’s location is the leading cause of noise in position data collected from the cellular network as they transfer calls to the nearest base station for traffic management, creating imprecise and overlapping Voronoi polygons (Chen et al., Reference Chen, Ma, Susilo, Liu and Wang2016). The time-based filter is used to ignore oscillation and agglomerative (hierarchical clustering) methods to extract truthful location data from raw CDR.

However, handling such sensitive data requires appropriate protocols to address concerns around data privacy. While anonymizing data is necessary, Kondor et al. (Reference Kondor, Thebault, Grauwin, Gódor, Moritz, Sobolevsky and Ratti2015) show that it is theoretically possible to identify users based on their mobility patterns alone (Kondor et al., Reference Kondor, Thebault, Grauwin, Gódor, Moritz, Sobolevsky and Ratti2015). It is, therefore, best practice to restrict access to individual observations and use aggregated indicators for the purpose of analysis.

2.3. Country context: The Gambia

The Gambia is a West African country of 2.3 million people surrounded by Senegal. The country has experienced prolonged spells of violence and instability and is currently undergoing a transition process to restore its democratic institutions. With a Gross National Income (GNI) of 740 USD (current, Atlas method), The Gambia is classified as a low-income country, with more than 10% of the population living in extreme poverty. The capital city region at the mouth of the Gambia river encompasses Banjul city and the Kanifing region, with tourist resorts strung southwards along the coast (see Figure 1). Tourism and the civil service are the largest drivers of formal employment. Inland is largely rural, its economy is driven by agriculture, and largely dependent on the flow of domestic and international remittances from migrants. These disparities in access to services and opportunities have led to high levels of internal migration, especially among the young who left rural areas to look for better jobs in the capital city region (The World Bank, 2020).

Figure 1.

Administrative boundaries of The Gambia. Source: Authors. Note: Names on the map indicate eight local government areas (LGAs). Boundaries present 48 Districts. LGA boundaries are highlighted in bold.

The Gambia confirmed its first case of COVID-19 on March 17, 2020. As an immediate response to prevent the spread of the disease, the government imposed a social-distancing policy on March 18. A state of emergency was declared on March 27 and extended on April 3 and May 19. In response to the spread of COVID-19 and the closure of international borders, the burgeoning tourism economy collapsed near the height of the tourist season, driving up unemployment. Many migrants returned to their home villages, creating an urban exodus. Trade and travel within the country were reduced to the strict minimum, as authorities enforced restrictions on movement. The period of analysis also includes Ramadan (April 23 and May 23), which, in this majoritarian Muslim country, is traditionally a time of reduced economic activity as many travel to be with family.

PURA is an autonomous government entity that oversees water, electricity, and telecommunication services in The Gambia. As part of its oversight activities, the regulator collects aggregated indicators from MNOs to monitor service quality. With technical support from the World Bank and the University of Tokyo, it has worked with mobile phone operators to expand the list of indicators routinely collected for the purpose of mobility analysis and store them in a secure on-site server. In this effort, it has collaborated closely with both The Gambia Bureau of Statistics (GBoS), and the Ministry of Health.

3.1. Securing institutional and organizational access

The analysis of CDR data in The Gambia goes back to a dialog among The World Bank, the GBoS, and the PURA to explore the use of big data to create an evidence base for policy and project design in the context of economic and social development. The Gambia experiences high levels of domestic and international migration, which provides access to opportunities and services and triggers a steady flow of remittances. In 2019, the share of emigrants relative to the total population was around 5%, and personal remittances were equivalent to 16% of GDP (World Development Indicators, 2020). As survey and administrative data were outdated, the three parties agreed to pilot the use of de-identified, anonymized, and aggregated CDR data to identify locations with high levels of outmigration and describe patterns of human mobility. Initially, this analysis was based on summary statistics of incoming and outgoing international calls on the level of cell towers which overlap with known hotspots of international migration. This use-case relied solely on de-identified and aggregated data and allowed the team to demonstrate a proof of concept while building trust with government counterparts. The spread of COVID-19 prompted interest in internal mobility, altering the development objective for this partnership.

A workshop in February 2020 was instrumental to initiate a partnership on “Big Data for Development” that evolved around ownership and sustainability. A joint vision and a clearly specified use-case to focus the analysis on internal migration helped to coordinate expectations and build capacity using a practical example. Moreover, a broader audience during the initial workshop—involving stakeholders from the private sector, government ministries and agencies, academia, civil society, and development partners—confirmed the demand and interest for access to and analysis of CDR data. During this discussion, it was instrumental to convince the MNO to join this initiative, as they are collecting and providing the CDR data. Their agreement was based on the idea that training would also enhance their in-house capacity to analyze CDR data in order to improve their business operations and enhance customer relations. The workshop also offered a forum to discuss any institutional, organizational, or technical challenges, and to showcase how other countries have managed these concerns.

Once COVID-19 hit The Gambia in March 2020, all parties agreed to revisit the focus of the collaboration and explore the use of CDR data to respond to the health and economic crisis brought on by COVID-19. In light of limited viral testing and health facilities equipped to handle rising numbers of COVID-19 patients, the government announced a national health emergency with profound restrictions on human mobility (Hale et al., 2020). As part of this dialog, it became clear that prolonged social distancing would bear a high cost for households and firms (Gottlieb et al., Reference Gottlieb, Grobovsek, Poschke and Saltiel2020), and there was interest to create an evidence base for smart containment measures. Based on successful applications in other countries, the analysis then focused on the use of CDR data to understand patterns of human mobility during COVID-19.

3.2. Strengthening technical capacity

In addition to building a consensus about the analysis of CDR data, partners agreed to strengthen technical capacity and ensure knowledge transfer. Crucially, rather than building a system from scratch, efforts were directed at strengthening existing data collection protocols between the MNO and the PURA to include the necessary indicators. As part of its mandate to monitor the quality of calls, PURA already had established a centralized repository of data, which was plugged into the respective MNOs systems and updated in real-time. After securing the necessary approvals, the team worked with the system administrator to include additional indicators as part of this routine monitoring for use in the analysis. This minimized the reporting burden on MNOs, facilitating compliance. To ensure an additional level of security, the data collected for this project was firewalled and stored on a separate server on the premises, with remote access strictly limited to key researchers and system administrators. Capacity-building in the preparation and analysis of CDR data also built trust and offered an opportunity to discuss lessons learned from other countries. It helped to establish a platform to continue the work during the following month when all interactions between the counterparts in The Gambia and the team of researchers shifted online.

Throughout the partnership, PURA played a crucial role in working closely with the MNOs to obtain access to the CDR data. The regulator for telecommunication services used its convening power to discuss data sharing with MNOs while upholding national and international standards for data privacy. Two out of four MNOs agreed to provide access to their CDR data. After training in one-way encryption using a 160-bit hash function, they provided anonymized data to the regulator. In accordance with national regulatory requirements, the regulator set up a secure file transfer protocol (FTP), and all data were stored on-premise on a dedicated server. Aggregation of the data was conducted through highly restricted remote access into the server, which constrained computational capacity but kept the data secure and confidential. The team working remotely only had access to aggregate indicators for analysis which precluded any possibility of de-anonymizing the data.

3.3. Hardware requirements

A Hadoop platform was introduced as the primary system for data processing and analysis. Hadoop is a set of open-source software for data-intensive and distributed applications aiming to solve massive amounts of data and computation. Multiple machines work together as a cluster with parallel computation distributed among nodes. At the limit of storage and processing time, a cluster can be easily scaled by adding more machines to the cluster.

For the hardware requirements, a minimum of four machines is necessary to build up a cluster (see Figure 2). One works as a master node to keep metadata and manage processing jobs. The other three machines work as slave nodes or storage and computation nodes. The network connection among nodes must be at least a gigabit of ethernet to ensure no bottlenecking occurs on data transfer. An additional machine can be added for visualization, anonymization, and jump host to the cluster, which is located in a separate network ensuring data security and accessibility. The hardware can also be a virtual machine or physical machine depending on the existing infrastructure and additional cost calculations. In The Gambia, the team started with virtual machines on pilot data to provide preliminary results, and then upgraded to full hardware with the full dataset.

Figure 2.

A Hadoop Cluster as a hardware solution to process CDR data. Source: Authors.

Allowing continuity and continual updating of the data, a well-defined data pipeline is essential. The data were provided by MNOs in a compressed or comma-separated file daily and uploaded to a secure FTP server under a private link or virtual private network. The task was run to extract data, import the Hadoop cluster, preprocess the data, and prepare of for analysis. The CDR data contain a rich set of information mainly used for network routing, usage accounting, and handset localization. The CDR Data consist of the following: the International Mobile Equipment Identity (IMEI), the International Mobile Subscriber Identity (IMSI) of the caller, a timestamp indicating when the session started, the usage duration, the base station identifier, and the activity type (call, short message service [SMS], and data communication). The base station ID is mapped onto the base station dataset according to the latitude and longitude of the cell tower. Ensuring privacy, the identifiable data fields such as the IMEI are encrypted and replaced with computer-generated random numbers before the analysis.

4.1. CDR data descriptive

The mobile penetration rate of The Gambia was 94.2% in 2013 and rose to 140% in 2018, which was higher than the average of developed countries (ITU ICT-Eye, n.d.). As of 2018, 98.4% of households reported ownership of at least one mobile phone with limited variation across regions and between rural and urban areas. On the individual level, 85.1% of men and 74.1% of women in the age group between 15 and 49 years own a mobile phone (The Gambia Bureau of Statistics, 2019). According to PURA, the four major MNOs provide services to 2.59 million subscribers as of 2020, even allowing for persons with multiple SIM cards. In this paper, we use CDR data for a 3-month period between March 2 and May 31, 2020, offering a snapshot of changes in mobility during the COVID-19 lockdown.

Data were made available for two major MNOs, which cover approximately 70% of the market and include around 1.75 million subscribers. On average, the data comprise 18.8 million data points per day with very limited variations over the data period, which amounts to 2 billion anonymized observations in total. Hence, we assume that cell phone usage in terms of transaction volumes did not change fundamentally once COVID-19 hit the country. The average number of records per subscriber per day is 10.6 where approximately 2.6 records are used for calling. Like other developing countries, the practice of using multiple SIM cards is common in The Gambia. We expect a certain overlap between the two MNO subscribers, which might have resulted in over-representing the multiple-SIM-card holders. In this study, the impact of multi-SIM holding on the analysis result is considered to be limited since the two MNOs primarily market to different socio-economic groups. One of them is a leading MNO in The Gambia and is popular in urban areas with high-speed internet services. The other MNO provides only voice and SMS with inexpensive plans, which are much more popular in rural areas.

The preparation and analysis of CDR data under this project are based on a protocol to address concerns of privacy and confidentiality. Raw CDR data include several identifiers associated with each record, such as phone number, IMSI and IMEI. We employ a three-stage approach to anonymize these identifiers in order to protect data privacy.

1. 1. First, identifiers are encrypted using a one-way function by the MNOs on their premises.

2. 2. Second, the encrypted identifiers are replaced with the random numbers after the data are combined. Lists of cell towers and the locations provided by the MNOs are pooled in the regulator’s premise, and cell-tower locations are clustered using Ward’s hierarchical clustering, with a maximum distance constraint of 1 km from the centroid of the cluster.Footnote ¹ We then use the centroid of the cluster to match and map the de-identified CDR data to their respective cell-tower locations. This process lowers the spatial granularity of cell-tower distributions.

3. 3. Third, the results of all indicators are aggregated at the administrative unit level. There are certain concerns about reverse engineering for the re-identification of de-identified CDR data (Kondor et al., Reference Kondor, Hashemian, de Montjoye and Ratti2018) but the abovementioned aggregation process lowers the risk of reverse engineering.

4.2. Key indicators for the analysis of human mobility

The analysis of patterns of human mobility during COVID-19 in The Gambia is based on a set of mobility indicators, which are calculated based on CDR aggregates. The indicators capture changes in population movements during the baseline, under COVID-19 and post-intervention periods, and results can be updated continuously as additional CDR data become available. We used the first 2 weeks of March before the lockdown as baseline to obtain indicators about routine mobility levels.

The standardized indicators were proposed by the World Bank COVID-19 Mobility Task Force and build on a framework developed by Flowminder to support MNOs in producing basic indicators from telecom data (see Flowminder COVID-19 Resources—Mobility indicators, n.d.). Methodologies for computing indicators were designed to minimize computational intensity in resource-scarce settings. The indicators are fully anonymous and contain no information about individual subscribers, ensuring that the privacy of subscribers is maintained at all times. They are robust to sparse tower distribution and to infrequent phone usage, both of which are common in low- and middle-income countries. Eleven key indicators provide proxies for population, location of residence, distances traveled, and daily mobility trends across regions at different geographic and time levels. For this project, we selected 4 out of the 11 indicators. Table 1 summarizes the selected indicators based on information on the World Bank COVID-19 Mobility Task Force repository on GitHub.Footnote ²

Table 1.

Summary of key indicators for the analysis ^a

^a Source: Authors’ adaptation based on the World Bank COVID19 Mobility Task Force repository on GitHub.

Abbreviation: SMS, short message service.

4.3. Application in The Gambia

This paper makes use of the proposed indicators from the World Bank COVID-19 Mobility Task Force and applies them to the country context of The Gambia to analyze patterns of human mobility during COVID-19. More specifically:

1. 1. Indicator 3 shows changes in the population distribution over time. As subscribers are accounted for in every region in which they use their phones, it overestimates subscribers who visit multiple regions in a day when computed at the administrative unit level. The value of this indicator can be also affected by load-sharing where several cell towers jointly cover a certain area due to network optimization. This impact is mitigated by the cell-tower clustering, which was described as part of data preprocessing in the previous section. We compute this indicator at the national level for examining how the number of active subscribers as a whole country changes over time and for adjusting the result of other indicators.

2. 2. Indicator 6 illustrates changes in the location of residency, which could infer the incidence of migration over the data period. For mapping the residential distribution, there are various methodologies and algorithms (Ahas et al., Reference Ahas, Silm, Järv, Saluveer and Tiru2010; Deville et al., Reference Deville, Linard, Martin, Gilbert, Stevens, Gaughan, Blondel and Tatem2014) which provide more accurate estimates compared to the proposed method. We consider it still useful for detecting a flexible home location reflecting weekly changes as the estimation result is used at the administrative unit level. In addition, the proposed method is relevant under resource-scarce settings as it is not computationally intensive.

3. 3. Indicator 7 demonstrates changes in levels of mobility over the data period. The value of this indicator is defined as the average distance traveled per person residing in a region. This indicator has limitations in detecting mobility particularly in rural areas due to lower cell-tower density. Mean values for regions in rural areas tend to be affected by extreme values generated from distant cell towers, which are much longer distance than can be traveled. In addition, median values for rural areas tend to be zero because short-distance travel is not detected when a wide area is covered by a cell tower. We use the value of the 75th percentile. It results in representing the mobility patterns of people whose mobility is relatively higher. However, the value enables us to capture changes without being affected by extreme values.

4. 4. Indicator 10 describes the sizes of population inflow and outflow. We use this indicator for examining changes in population inflows. This indicator can be used for constructing OD matrices but has limitation in capturing long-distance trips. This is because a trip for constructing an OD matrix is defined by each consecutive pair of records, meaning that a long-distance trip is transformed into a set of several short trips, and thereby a link between the origin and destination of the long trip is missed.

The four indicators above are selected for application in The Gambia to inform COVID-19 responses. These indicators are useful for capturing changes in mobility patterns, occurring in a relatively short period in response to mobility restrictions, and for understanding the mobility patterns, directly affecting the spread of infectious diseases. We highlight that the use of Indicator 3 is critical for mitigating the impact of changes in active subscribers over time. Overall, the proposed methodology for the indicators is relevant to COVID-19 particularly in resource-scarce settings as it enables the generation of actionable statistics even with limited capacity and computing resources, which is a common state in many developing countries.

The following section summarizes key findings based on the set of four indicators outlined above and demonstrates how mobility statistics produced from CDR data can be used for understanding dynamic changes in population distribution and movements.

6.1. Policy dialogue: first, find a use-case

This use-case highlights that CDR data are particularly useful in countries with limitations to frequency, timeliness, and coverage of administrative and survey data. Although the mobility statistics have constraints in terms of capturing all aspects of human mobility, our results show that statistics produced from CDR data capture changes in population distribution and movements, which continue to vary in a short period. This is particularly useful during an emergency like COVID-19, where traditional data collection methods may be too slow to capture the rapidly evolving situation.

The successful implementation of this use-case on COVID-19 is based on an early engagement with PURA and GBoS, which also established a platform to strengthen the policy relevance of the analysis. The workshop in February 2020 offered an opportunity to discuss with stakeholders their ideas, expectations, and concerns regarding the use of CDR data. Furthermore, building consensus among all stakeholders and using strategic alliances and champions embedded in the country dialog helped to foster ownership and sustainability of the project. When the COVID-19 crisis struck, the groundwork was already in place, and the team could quickly produce analytics focused on the impact of COVID-19 on patterns or human mobility.

6.2. Engagement model: bring decision-makers on board

As results from the use-case on COVID-19 became available, PURA and GBoS used these findings to inform their participation in the government task force for COVID19. The early dissemination of results helped to inform decision-makers and prompted requests for a scaled-up version providing real-time data during a quickly evolving health and economic crisis. Unfortunately, efforts to quickly turn this case-study into a fully functional data pipeline were delayed due to constraints in implementation capacity until late 2020. However, once the necessary hardware and training were provided, the CDR data pipeline became operational in early 2021.

Throughout the dialog with decision-makers, presentation of findings in an easily accessible format and identification of specific policy recommendations strengthened the support and the interest in the project. Maps created an entry point for dialog with a technical and non-technical audience as they were visually appealing but still contained important lessons (see Figure 8). Moreover, interpretation of results and specific applications in the context of the pandemic supported communication. For instance, the team argued that findings from the use-case on COVID-19 could inform targeted testing initiatives, by concentrating efforts in areas of high mobility. When a full lockdown is not possible given the economic costs, this can also inform where social-distancing policies should be enforced given higher mobility and associated with these increased risks of transmission. In addition, results demonstrated that the lockdown disproportionately affected urban areas by restricting economic activity, and relief and recovery efforts should therefore aim to address these adverse effects. These are but a few of the policy-relevant insights CDR data can deliver.

Figure 8.

Weekly averages of distances traveled at the district level—ratio to the baseline. Source: Authors’ calculations.

6.3. Big Data for development: partnership, innovation, and capacity

The partnership on “Big Data for Development,” and the analysis of CDR data in the context of COVID-19, highlight that real-time data and analysis are valuable, but only when produced in close collaboration with local counterparts. Rather than aiming for shortcuts, the project brought together a statistics agency (GBoS) and government regulator (PURA), two entities who rarely collaborated in the past. In its engagement, the project also sought to play to the respective strengths of its counterparts. PURA brought the regulatory mandate and technical capacity to collect and process the data, while GBoS contributed in guiding and motivating the analytics. Positive feedback from other government entities, including the Ministry of Finance, has created incentives for GBoS and PURA to continue exploring opportunities to collaborate and innovate.

Through this project, the team introduced its counterparts to an innovative approach to handling big data while also following strict protocols on how to preserve the privacy and confidentiality of this new type of information. Recommended privacy practices include establishing appropriate encryption, maintaining file transfer and storage protocols to ensure the security of highly confidential data, and ensuring adherence to regulatory requirements and best practices. This sort of direct engagement model fosters innovation and ownership, helping to build capacity through continued engagement. This included training the MNO operators in one-way encryption protocol, and installing a stand-alone server on PURA premises to hold the data, with strict remote access protocols.

Finally, working in a limited-capacity context required flexibility, and from time-to-time involved compromises. In terms of hardware, an initial server provided to PURA for piloting purposes became the go-to for data storage when COVID-19 impeded the acquisition of additional server capacity. In terms of analytics, although there was a tool for producing standardized mobility statistics available on the GitHub repository, we chose to write our own script. This was to accommodate the system parameters on PURA’s premises. Since much of the analytical work was done through remote access, it was restricted by network capacity and could be interrupted by electrical outages. This required breaking computationally-intensive tasks into multiple steps with intermediate results, so that any interruption would only disrupt the current computation and not the whole script.