3.1. Benefits of e-GP

There are multiple, tangible benefits associated with e-GP:

• • An improvement in workflow efficiency of procurement agencies. A reduction of 55 days in the tender cycle time was observed in India (Asian Development Bank, 2013). In South Korea, the duration of the processing of bids, from the receipt to the validation to the selection of the winning bid, was brought down from an average of 30 hours to 2 hours (Luijken & Martini, Reference Luijken and Martini2014). The adoption of the COMPR.AR system in Argentina led to strong efficiency gains: on average, the duration of the public procurement process fell by over 11 days (De Michele & Pierri, Reference De Michele and Pierri2020). BAObras, a public transparency platform for the procurement of over 1100 public works projects for a planned US$3.5bn facelift for the city of Buenos Aires in Argentina, reduced the time taken to share data by the city’s authorities by 93% (Open Contracting Partnership, 2021b).

• • A decrease in advertising costs. In the Philippines, there was P564 million savings in advertising; likewise in India, a decrease in advertising costs of $0.56 million (Bombay, Reference Bombay2011; Asian Development Bank, 2013).

• • An increase in the number of bidders. Because information and transaction costs in traditional paper-based procurement processes are more onerous for smaller firms, electronic procurement systems, which reduce such costs, attract greater participation. In Bangladesh, the rate of contracts awarded on a tender with only one bidder practically halved, dropping from 33% to 17% (World Bank, 2020a). In Karnataka State, India, the number of suppliers increased from 130 to 4800 in the first 3 years of operation (Ojha & Pandey, Reference Ojha and Pandey2014). In South Korea, the number of bidders doubled from 70,000 to 147,000 in the first 3 years (Cho & Byeon, Reference Cho and Byeon2004). An analysis of the open data from Ukraine’s ProZorro e-GP reforms from 2021 shows that a remarkable 97.7% of businesses (some 64,801 by number) bidding in ProZorro are Small and Medium Enterprises (SMEs). In 2021, they won 395,271 contracts for $16.6 billion, while large companies (some 1524 entities) won 20,249 contracts for $5.99 billion.

• • Better oversight of suppliers and improvement in service delivery. In Indonesia, 15% more construction projects were completed on time (Lewis-Faupel et al., Reference Lewis-Faupel, Neggers, Olken and Pande2016). In India, there was a 12% increase in road quality grade (Lewis-Faupel et al., Reference Lewis-Faupel, Neggers, Olken and Pande2016). e-GP reforms in Paraguay from 2015 reduced amendments to contracting processes from 19% in 2013 to just 3% in 2016. Improved monitoring meant that more than 80% of the schools most in need now receive funding, compared to fewer than 20% in 2015 (Open Contracting Partnership, 2017).

• • Increased transparency. Bauhr et al. (Reference Bauhr, Czibik, Fazekas and de Fine Licht2017) found that publishing five more pieces of information about each tender would save Europe up to 3.6 billion Euros. Furthermore, disclosing the information proactively rather than retroactively was important and had a bigger effect.

• • Better preparedness for emergencies, especially in coordinating a response to sudden marker shocks. A World Bank Survey covering 103 countries between April and August 2020 found that countries with comprehensive e-procurement systems were able to more promptly adjust their public procurement functions to respond to the COVID-19 emergency (Cocciolo et al., Reference Cocciolo, Di Maro, Samaddar, Bandiera, Bosio and Spagnolo2021). The Open Contracting Partnership (2022) detailed how availability of good quality open data on who was buying from whom improved emergency coordination, training, and monitoring in countries such as Chile, Colombia, Ecuador, Lithuania, Moldova, Paraguay, and Ukraine.

• • Implementation of red flags to detect and act on corruption and other risks. Availability of standardized, structured, machine-readable data in e-GP systems allow for proactive monitoring and identification of corruption, collusion, or other risks in contract planning, tender, and award processes, as well as automatically flags such concerns for action by the procurement authorities and control authorities. Ukraine’s ProZorro open procurement reforms are a best practice example of this with a sophisticated business intelligence engine using the data and resulting in hundreds of investigations and actions.

e-GP is an anti-corruption strategy. The direct costs of corruption include the misallocation or inefficient use of public funds, delayed projects, and/or lower quality infrastructure. The European Parliamentary Research Service (2016) found that a one-unit increase of the Corruption Risk Index raises prices (or reduces cost savings) on average by about 15%. They predict that the implementation of a full e-procurement system could reduce the costs of corruption risk in public procurement by around €924 m annually, which corresponds to a reduction of almost 20% of the current costs.

e-GP reduces corruption in procurement through several channels:

• • It lowers the bidding and transaction costs for firms, thus attracting more firms to the market, which stimulates competition and mitigates the high prices associated with single bidders.

• • The automation of certain functions reduces the potential for bribery and graft that exists with human interaction.

• • The digital fingerprint that is left from operating the system increases accountability in decision-making.

• • The transparent nature of the technology facilitates audit and monitoring functions and significantly lowers the cost of doing so. Importantly, red flags can be automated given the authorities (and sometimes even companies and citizens’) advance warning of a problem that can be investigated and fixed before things go seriously wrong.

An immediate and well-documented consequence of these channels is the lower prices paid by procuring entities. For example, Open data e-GP reforms in Moldova resulted in 15.4% savings on medical procurement transactions worth US$60 million and 19% savings on the HIV/AIDS program procurement budget, including 95% savings on key antiretroviral drugs when switching to a low-cost alternative dose (Open Contracting Partnership, 2021c).

Not all reforms result in a measurable cost impact, but other important factors, such as the quality of works and services, may still improve or delays may be reduced (Lewis-Faupel et al., Reference Lewis-Faupel, Neggers, Olken and Pande2016).

The multiple benefits mentioned above notwithstanding, the only benefit used in this cost–benefit analysis is the average percentage decrease in procurement prices, a benefit for which information was obtainable and comparable across several countries.

Table 4 lists country cases of cost savings realized as a result of e-GP implementation.

Table 4.

Effect sizes.

Note: Based partly on literature review by World Bank (2007a) and Fazekas and Blum (Reference Fazekas and Blum2021). Effects are national in scope, unless otherwise indicated.

Ukraine is an example of how much money can be saved from e-procurement. The country’s ProZorro reforms mean that its open architecture e-GP uses a reverse auction method for awarding contracts with open bidding as well as a real-time, open-data dashboard on the award data. This means that it is possible to calculate savings on awards as well as on the budgeted price that government was willing to pay to award the contract. Over the life of the reforms from 2015 to date (2022), savings have been at least US$1 billion a year. Furthermore, as expected, as agencies have become better at setting the prices for awards from the start, savings have come down from almost 15% in 2015 to about 3% for 2021.

After eliminating the extreme (highest and lowest) estimates from the sample above, the average percentage decrease in procurement price is 6.75%. This effect size is used to calculate a moderate estimate of the average decrease in procurement prices. The conservative effect size of 3%, the lower bound of the evidence reviewed, is also used in the scenario analysis.

The percentage of GDP spent on public procurement was 11.3% and 14.1% in low- and lower middle-income countries (Wu Chebili et al., Reference Wu Chebili, La Cascia, Collineau, Salomon, Calvet and Moreau2022). Using the Copenhagen Consensus Center’s GDP projections for 2022 and onwards, the average low-income country has a GDP of approximately $22.1 billion, whereas that of a lower middle-income country is $155.9 billion at the start of the intervention (see Table 5). The significant disparity in economy size and procurement volumes between the two income groups will prove to be one of the determining factors of the price reduction benefit from the e-procurement system.

Table 5.

GDP and procurement market projections.

The size of public procurement is $2.5 billion in low-income countries and $22 billion in lower middle-income countries, on average. Working with the assumption that only half of the awarded contracts go through the e-GP system in countries that have themFootnote ¹, the respective market sizes are reduced to $1.25 billion and $11 billion, respectively. However, the average of the income groups masks the variance in procurement market sizes that exist within the income groups: the bottom quintile of low-income countries as measured by GDP has an average procurement market of $500 million; its counterpart in lower middle-income countries is valued at $4.4 billion. This variance and its impact on the benefit–cost ratio are addressed in the scenario analysis below.

Procurement agency uptake and integration into the e-procurement platform are not instantaneous. Uptake at the start of the Operations phase (Year 5), based on the sample of countries, is 25.3%, on average. Hence, it is assumed some benefits do accrue during the Pilot phase, or Years 3 to 5. Therefore, there is gradual phasing of benefits beginning in Year 3 through to Year 12 (see Table 6).

Table 6.

Schedule of benefits, with average effect size 6.75%.

3.2. Costs of e-GP

Generally, the implementation of e-GP involves (World Bank, 2007a):

• • A comprehensive review of the whole public procurement system, procurement processes, procurement regulations, procurement market evolution, and whole-of-government reform processes. An evaluation of readiness must be undertaken for at least five elements: leadership, functionality, technical design and standards, private sector activation, infrastructure, and web services.

• • Legal and regulatory reform. For example, under Egyptian procurement law, tender announcements had to be published in two widespread newspapers. It had to be amended to include the internet. Likewise, supplier bids were identified by supplier stamps and can now be identified by e-signature.

• • User-centered research and design both of the procurement system and of the “business logic” of how to plan, award, and implement contracts effectively in a digital environment.

• • Mapping of the data, data standards, and decision-making across the procurement process, a publication policy of what data to share, with whom, and when, and the performance indicators to be tracked internally and externally.

• • A reorganization of back-office functions to deliver an improved, digitized business process: ordering, payments, catalog maintenance, among other functions to ensure clarity of the procedure and responsibilities.

• • The acquisition and configuration of the specific technological and the data support skills needed for e-GP implementation.

• • User engagement, training, and outreach across the e-GP implementation process.

There are costs associated with each of the implementation phases. The Planning and Design and Build phases incur investment costs. These costs are modeled over a 29-month period; 11.1 for the former, 18 for the latter, running into Year 3 (see Table 7).

Table 7.

e-GP costing by country case.

Note: Costs are inflation-adjusted to 2020 depending on year of occurrence. ‟n/a” implies that the phases were not readily distinguishable.

Often overlooked is the fact that e-procurement tools may also increase transaction costs by introducing new types of costs such as system design rigidity (i.e., not being able to accommodate certain atypical cases) and IT system maintenance, data storage, and vendor lock-in from specific technical solutions. This constitutes what is referred to as a one-off transition cost for users that can be substantial in many developing countries (Fazekas & Blum, Reference Fazekas and Blum2021).

Also in Year 3 is the start of the Pilot, lasting 29.3 months, on average, and which is assumed to incur annual operations and maintenance costs. Ongoing maintenance includes data security and management and training, including both relevant government personnel and firms. Annual operations and maintenance costs are also assigned to Years 6 through 12, with a 2% annual cost increase (Table 7).Footnote ²

The value of the training component should not be underestimated nor underbudgeted. According to the Asian Development Bank (2013), the true value a government receives from e-GP is in the size of its online marketplace and the level of competition generated by its user base. Building the user base does not happen overnight. It requires a great deal of effort in the promotion, marketing, and delivery of an e-GP system and procurement reforms that encourage vendors to participate. Therefore, external training, outreach, and IT support to suppliers are also required (World Bank, 2007a; Asian Development Bank, 2013; Fazekas & Blum, Reference Fazekas and Blum2021) and have been a key component in successful e-GP reforms (see, e.g., Open Contracting Partnership, 2016). These costs form part of the operations and maintenance costs.

Maintaining the security of an e-GP system is also expensive and often demanding. In a custom e-GP platform, the government usually manages the infrastructure alone, in addition to training users and internal developers, which represents a major investment. Functionalities requiring specific attention where it relates to data security include e-tendering, e-reverse auctions, e-evaluation/awarding, contract management, e-purchasing, and e-registration (Wu Chebili et al., Reference Wu Chebili, La Cascia, Collineau, Salomon, Calvet and Moreau2022).

According to an evaluation report of Vietnam’s e-GP system, a full-fledged off-the-shelf e-GP system developed using open source programming platform and database as well as with open standard architecture with required customization can be bought for between $1.2 and $1.5 million, depending on the scope of customization (Shakya, Reference Shakya2016). However, this estimate does not include the other costs referred to above, which include implementation costs; content aggregation, rationalization, and maintenance; catalog/search engine; transactions; end-user training; process re-engineering; associated licensing—for example, additional DBMS fees, integration-ware licensing, and marketplace participation (World Bank, 2007a). With these costs taken into consideration, the World Bank (2007a) estimates that Italy’s e-GP system costs 10.4 M Euros in 2002, and it was 4 years before the system reached full operability.

Ukraine’s ProZorro reforms provide a robust recent costing of an open-source reform effort including a full estimate of each category as it was subjected to a detailed review specifically to cost out the reforms using independent experts by the World Bank in 2017. The more costly activities of the implementation process appear to be related to human resources (60%), whereas the hardware costs were only 10% of total implementation costs (see Figure 1).

Figure 1.

Breakdown of ProZorro program costs by cost input category. Source: Results for Development (2017).

Wu Chebili et al. (Reference Wu Chebili, La Cascia, Collineau, Salomon, Calvet and Moreau2022) undertook a survey of software developers in order to develop costing scenarios for implementation of e-GP in African countries. For a full-service e-GP platform (including e-procurement planning, e-publication/notification, e-tendering, e-evaluation/e-awarding, e-registration, vendor management, and procurement monitoring and reporting), the highest cost scenario, as customized for the country, was $4.2 million. The medium cost scenario of a commercial off-the-shelf system was priced at $3.4 million. The lowest cost scenario is an off-the-shelf system called SaaS, a shared service, made available over the Web with no hosting or installation required and customization is limited to software capabilities. This option is considered the most relevant for e-GP by software developers, and the total cost is $2.6 million. Five years is considered the average implementation period.

Investment costs are, on average, $8.5 million and are incurred during the Planning and Design and Build Phases, Years 1 to 3 (see Table 8). The development and implementation of South Korea’s KONEPS are one of the highest cost estimates found in the literature. KONEPS is fully integrated and deployed across all procurement agencies as well as procurement-related external agencies like the Ministry of Finance. Furthermore, it was an end-to-end system in which higher functionalities like encryption technology and digital certification were integrated from the start. Finally, KONEPS is internally operated and maintained by the Public Procurement Service, requiring the acquisition of hardware, software development, and human resources capacity development, rather than the outsourcing option chosen by other countries (Cho & Byeon, Reference Cho and Byeon2004; OECD, 2016).

Table 8.

Schedule of costs, USD million.

Note: The total unadjusted cost of e-GP, over 12 years, is $22.3 million, which includes anticipated annual cost increases of 2%. When discounted by 8%Footnote ³, the net present value of costs is $16.7 million.

Average annual maintenance costs are $1.1 million and are incurred throughout the Pilot and Operations phases, beginning in Year 3 through to Year 12. They are driven up by estimates from the Philippines and the government of Karnataka, India. Facing human resource capacity constraints, the former opted for a hybrid approach in which the Procurement Service outsourced the back-end system to a third party. It was a whole of government conversion, including national government agencies and state universities and colleges. Other maintenance costs include procurement staff and management training (Bombay, Reference Bombay2011). Likewise, in the latter, the government selected a state-wide, end-to-end system (including supplier registration, e-tendering, contract management, catalog management, e-auction, e-payments, among others), covering more than 70 departments, with the potential to scale horizontally, and chose to outsource all aspects of implementation, operations, and maintenance to a private sector partner for 5 years. Their maintenance costs also included a help-desk service for system users, two training facilities for procurement officers, which operated for 6 years, and an “Expert Cell” which managed the Public Private Partnership (PPP) arrangement (Ojha & Pandey, Reference Ojha and Pandey2014). It is worth recognizing that governments already have a cost with respect to maintaining whatever current system and regime exist, even if it is on paper, and it is not clear that the maintenance costs, as opposed to the conversion costs, of a more digitized system are more. Nonetheless, we have budgeted for additional costs.

Since implementation and maintenance costs depend largely on the choice of software and the degree of customization, as well as the business model chosen to manage the system, it is assumed that both low- and lower-income countries will face the same options and will make choices based on their local context and needs, which do not necessarily vary by size of the economy.

3.3. Cost–benefit analysis of e-GP

The central estimates for return on investment in typical low- and lower middle-income countries are 38 and 309, respectively (see Table 9).

Table 9.

Cost–benefit ratios, USD millions.

Note: Monetary values are discounted at 8%.

Results are highly sensitive to parameters relating to effect size, the percentage of procurement that goes through the e-GP system, the length of the implementation phases, and the size of the country’s economy. Tables 10–14 show the benefit–cost ratio of various scenarios.

Table 10.

Sensitivity analysis on effect size.

Table 11.

Sensitivity analysis on duration of implementation phases.

Table 12.

Sensitivity analysis on e-GP penetration.

Table 13.

Sensitivity analysis on size of economy.

Table 14.

Sensitivity analysis of failed rollout.

Of the 11 country cases, the savings in procurement prices documented ranged from 3 to 25%. The average is considered to be the moderate estimate, a 6.7% reduction in procurement prices was found in half; the conservative estimate of 3% was the minimum estimate found in 2 cases, Ukraine and Chile. Finally, the optimistic estimate of 10% could be found in 10 studies.

For two countries, the phases were not distinguishable, leaving a sample of nine. The time from inception to launch (Planning, Design and Build and Pilot) spanned from 22 (Ukraine) to 90 (Philippines) months. Four of the countries completed the process in less than 58 months (South Korea, Italy, Rwanda, and Ukraine).

Of the six countries for which we were able to obtain the percentage of public procurement that goes through the e-GP system, the average was 75%. However, examination of the Global Public Procurement Database revealed that the indicator was more likely to be around 50%, which is treated as the moderate estimate.

The size of the economy, and hence the size of the procurement market, greatly limits the benefits of this intervention. The lowest quintile of low-income countries includes Somalia, Sierra Leone, Burundi, and Liberia. These countries can expect a benefit–cost ratio of 8 versus the average of 39. The lowest quintile of lower middle-income countries includes Cambodia, Senegal, El Salvador, and Honduras, and these countries can expect a benefit–cost ratio of 64 against the average of 319, all other things being equal.

Finally, there is also the scenario in which rollout of the e-GP platform fails after the pilot and a second investment is needed to re-launch it, which typically includes an upgrade of the system if several years have lapsed since the pilot. There are also observed cases in which an e-procurement system is replaced/upgraded before widespread adoption across procuring entities. The example of this is Moldova, which is about to launch its 3rd e-procurement platform in less than 10 years with gaps between implementation of 2 systems, while not yet achieving full implementation.

Table 14 shows the benefit–cost ratios for the scenario in which there is a re-investment after month 59. The benefit–cost ratios of 27 and 198 represent the returns after a failed initial rollout.

The costs and benefits of the private sector, namely bidding firms, have not been included in the analysis. These refer to technology and hardware acquisition and adjustment costs and the opportunity costs of sending personnel to training sessions, among other costs. Benefits include workflow efficiencies like reduced time and travel when interacting with government officials, lower marketing and business development costs, improved cash flow, and a greater number of government contracts.

Furthermore, there are other tangible benefits of e-GP that have been documented in the literature. These include the reduction in the amount and frequency of bribes paid per government contract, as well as the reduction in advertising costs. The reduction in expenditure in one segment of the economy represents a savings in another. Essentially social transfers, they would have limited effect on the cost–benefit analysis.

The Copenhagen Consensus Center has in the past undertaken cost–benefit analyses of digitization of functions that alter the nature and form of the private sector’s interaction with public officials. The benefits far outweigh the firms’ costs of adjusting to new regulations. E-filing coupled with tax nudges in Malawi had a benefit–cost ratio of 7; digitization of property and business fees functions in Ghana, 9; the automation of VAT collection in Bangladesh, 6; and digitizing the whole of government in Haiti, 5.

Finally, the excellent benefit–cost ratios are ultimately contingent on the successful implementation and compliance to the system. Sometimes, government IT projects do fail; in such cases, the benefit–cost ratio would be less than 1.

These results give plentiful motivation for governments wishing to optimize public expenditure and reduce the likelihood of corruption at the same time. There are, however, a number of critical success factors to maximize the anticipated benefits of the intervention.