1.1 The importance of pluralism in international science–policy platforms

Achieving global sustainability requires tapping into diverse sources of knowledge (Cash et al., Reference Cash, Clark, Alcock, Dickson, Eckley, Guston, Eckley, Guston, Jäger and Mitchell2003; Tengö et al., Reference Tengö, Brondizio, Elmqvist, Malmer and Spierenburg2014). Knowledge pluralism, understood as the integration of diverse knowledge and knowledge systems (White & Lidskog, Reference White and Lidskog2023), can be epistemic and geographic. Epistemic pluralism refers to the different ways in which the world and our relations to it are conceived, and how our understandings about it are generated (Gardner, Reference Gardner2013). Achieving epistemic pluralism implies integrating knowledge from different theoretical perspectives and knowledge systems, such as scientific and lay knowledge. It is instrumental to understand complex socio-ecosystems: for instance, traditional ecological knowledge, held by Indigenous Peoples and Local Communities (IPLCs), can provide crucial insights on ecosystem processes and functions (Berkes et al., Reference Berkes, Colding and Folke2000); it also forms the basis of local resource management practices that contribute to biodiversity conservation and to the resilience of socio-ecological systems (Mavhura et al., Reference Mavhura, Manyena, Collins and Manatsa2013; Turnhout et al., Reference Turnhout, Bloomfield, Hulme, Vogel and Wynne2012). Combining diverse sources of knowledge, including both academic and non-academic knowledge, can therefore enable the achievement of the United Nations (UN) sustainable development goals (Cash et al., Reference Cash, Clark, Alcock, Dickson, Eckley, Guston, Eckley, Guston, Jäger and Mitchell2003; Díaz-Reviriego et al., Reference Díaz-Reviriego, Turnhout and Beck2019). Geographic knowledge pluralism refers to the integration of knowledge that is produced in diverse socio-ecological contexts. The world is a mosaic of different ecosystems, cultures, and natural resource management practices. These diverse socio-ecological contexts imply different decision-making practices, ecological challenges, values, and interests. Combining literature focusing on these diverse socio-ecological systems is therefore necessary to understand how different sustainability or conservation strategies may fare differently in different contexts (Cash et al., Reference Cash, Clark, Alcock, Dickson, Eckley, Guston, Eckley, Guston, Jäger and Mitchell2003; Merçon et al., Reference Merçon, Vetter, Tengö, Cocks, Balvanera, Rosell and Ayala-Orozco2019).

International science–policy platforms (ISPPs) synthesize state-of-the-art knowledge on specific topics (e.g. climate change or biodiversity loss) and translate the insights gained into policy-actionable knowledge. Their role is to serve as an ‘honest [knowledge] broker’ who informs decision-makers about the range of options available to them and the implications of each option (Pielke, Reference Pielke2007). Given the existence of contrasting perspectives, values, and interests informing the scientific processes of ISPPs, epistemic and geographic pluralism are necessary to ensure that diverse perspectives reflecting a wide range of worldviews and normative positions are considered (Carrier, Reference Carrier2013; Díaz-Reviriego et al., Reference Díaz-Reviriego, Turnhout and Beck2019; van den Hove, Reference van den Hove2007). Moreover, plural perspectives on diverse policy options can help policymakers adapt scientific insights to their particular contexts, and to consider uncertainty in the decision-making process (Pielke, Reference Pielke2007). Achieving pluralism, particularly in the inclusion of both academic and non-academic knowledge from relevant stakeholders such as IPLCs, can also help increase the legitimacy of ISPPs (Kohler, Reference Kohler2022; Timpte et al., Reference Timpte, Montana, Reuter, Borie and Apkes2018). For instance, some governments, particularly from the Global South (e.g. Bolivia, Ecuador, and Cuba), have been critical of the promotion (or hindrance) of particular policies by ISPPs claiming that they could interfere with their own national biodiversity conservation strategies (Sala & Torchio, Reference Sala and Torchio2019), and that they inadequately represent the challenges they experience and their perspectives on how to address them. For these reasons, one major challenge for ISPPs is to produce assessments that present diverse views from different regions, cultures, and knowledge systems. Addressing this challenge requires overcoming the biases that characterize contemporary scientific production.

1.2 Biases in scientific production and reviews

Subjectivity permeates all stages of scientific activity, including selecting an object of study, setting the scope of the research, and selecting research methods and analytical tools, as well as the research writing stage, where scientists tend to selectively report results (Cairo et al., Reference Cairo, Green, Forsyth, Behler and Raldiris2020; Zvereva & Kozlov, Reference Zvereva and Kozlov2021), and the publishing stage, where positive results tend to be favored over null or negative ones (Ekmekci, Reference Ekmekci2017; West & Bergstrom, Reference West and Bergstrom2021). The idea of ‘neutral knowledge’ is therefore an oxymoron (Borie et al., Reference Borie, Mahony, Obermeister and Hulme2021). By framing, researching and assessing the issues at stake in particular ways, experts implicitly legitimize certain claims and policy options over others (Dryzek, Reference Dryzek1997; Luke, Reference Luke1995; Wesselink et al., Reference Wesselink, Buchanan, Georgiadou and Turnhout2013).

Ensuring epistemic and geographic pluralism in assessments of the literature (such as systematic literature reviews or assessments produced by ISPPs) is therefore essential to avoid biases and to gain insights on the diversity of views and knowledges on a given topic. Yet, both epistemic and geographic biases have been noted in the literature. Scientific publishing has been shown to be biased against researchers from the Global South (Corbera et al., Reference Corbera, Maestre-Andrés, Calvet-Mir, Brockington, Howe and Adams2021; Maas et al., Reference Maas, Pakeman, Godet, Smith, Devictor and Primack2021; Nagendra et al., Reference Nagendra, Bai, Brondizio and Lwasa2018; Tuyisenge et al., Reference Tuyisenge, Kayitete, Tuyisingize, O'Malley, Stoinski and van der Hoek2023) as a result of multiple structural inequities, including reduced funding availability and scarce representation in editorial boards, as well as more stringent editorial and peer-review processes for researchers from the Global South (Yousefi-Nooraie et al., Reference Yousefi-Nooraie, Shakiba and Mortaz-Hejri2006). This may be the result of an imperialist heritage of science, which delegitimizes conceptual frameworks and ways of knowing and communicating that stray from modern scientific procedures promoted in the Global North (Mignolo, Reference Mignolo2009). In parallel, systematic literature reviews and other syntheses of knowledge tend to focus on academic knowledge, particularly from the natural sciences. This may be a result of different publication traditions: authors from different disciplines tend to publish in different outlets (for instance, scientific articles and books) some of which are difficult to access and for which publication rules vary. Further, gray literature (for instance, policy documents or reports from non-governmental organizations) is rarely considered in systematic reviews (Cairo et al., Reference Cairo, Green, Forsyth, Behler and Raldiris2020; Paez, Reference Paez2017; Yoshida et al., Reference Yoshida, Sitas, Mannetti, O'farrell, Arroyo-Robles, Berbés-Blázquez, González-Jiménez, Nelson, Niamir and Harmáčková2022) and sources of knowledge from IPLCs (e.g. declarations, stories, songs, poems, etc.) are often overlooked or contested by western scientists and institutions (Smith, Reference Smith1999; Tengö et al., Reference Tengö, Brondizio, Elmqvist, Malmer and Spierenburg2014).

Linguistic bias is of particular concern as it may hinder both geographic and epistemic pluralism. Non-native English speakers tend to spend more time producing and communicating knowledge in English (Amano et al., Reference Amano, Ramírez-Castañeda, Berdejo-Espinola, Borokini, Chowdhury, Golivets, González-Trujillo, Montaño-Centellas, Paudel, White and Veríssimo2023), which represents an important cost when engaging in ISPPs using English as the working language. Non-native English speakers may also struggle to publish in English (Ramírez-Castañeda, Reference Ramírez-Castañeda2020) and thus often choose to publish in their native language. In addition, systematic reviews and scientific searches are often conducted in English only (Droz et al., Reference Droz, Brugnach and Pascual2023; Ekmekci, Reference Ekmekci2017; Nuñez & Amano, Reference Nuñez and Amano2021), as a result of the limited resources of English-speaking research teams to translate studies written in other languages (Neimann Rasmussen & Montgomery, Reference Neimann Rasmussen and Montgomery2018). In parallel, search engines have been shown to favor literature written in English even in multilingual searches (Rovira et al., Reference Rovira, Codina and Lopezosa2021). This biases the scientific state-of-the-art, as this limits the diversity of information that is included in the reviews. Yet, non-English science contains crucial insights, for example, regarding biodiversity and conservation across diverse regions of the world (Amano et al., Reference Amano, González-Varo and Sutherland2016, Reference Amano, Berdejo-Espinola, Christie, Willott, Akasaka, Báldi, Berthinussen, Bertolino, Bladon, Chen, Choi, Bou Dagher Kharrat, de Oliveira, Farhat, Golivets, Hidalgo Aranzamendi, Jantke, Kajzer-Bonk, Kemahlı Aytekin and Jennions2021; Egger et al., Reference Egger, Zellweger-Zähner, Schneider, Junker, Lengeler and Antes1997; Konno et al., Reference Konno, Akasaka, Koshida, Katayama, Osada, Spake and Amano2020). Failing to better incorporate literature in languages other than English limits our understanding of complex environmental issues across different regions (Amano et al., Reference Amano, Berdejo-Espinola, Christie, Willott, Akasaka, Báldi, Berthinussen, Bertolino, Bladon, Chen, Choi, Bou Dagher Kharrat, de Oliveira, Farhat, Golivets, Hidalgo Aranzamendi, Jantke, Kajzer-Bonk, Kemahlı Aytekin and Jennions2021; Droz et al., Reference Droz, Brugnach and Pascual2023; Konno et al., Reference Konno, Akasaka, Koshida, Katayama, Osada, Spake and Amano2020; Lynch et al., Reference Lynch, Fernández-Llamazares, Palomo, Jaureguiberry, Amano, Basher, Lim, Mwampamba, Samakov and Selomane2021). These biases have to be taken into account and remedied in the work and implementation guidelines of international ISPPs.

1.3 Efforts and challenges toward pluralism in Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services assessments

One of the main tasks of the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services (IPBES) is the production of assessment reports that synthesize the state of knowledge regarding nature and its contributions to people worldwide. A unique feature of IPBES, relative to other environmental ISPPs (such as the International Panel on Climate Change), is its commitment to achieving geographic and epistemic pluralism (Borie et al., Reference Borie, Mahony, Obermeister and Hulme2021). IPBES strives to guarantee equitable regional representation in all its governing bodies, taskforces, and author teams. Decisions regarding the work of IPBES are taken by consensus in plenary where all (currently 144) member states are represented and have an equal vote. IPBES rules of procedures suggest the teams of experts producing IPBES deliverables must be balanced in terms of gender, region of origin, and disciplinary representation (Borie et al., Reference Borie, Mahony, Obermeister and Hulme2021; Tengö et al., Reference Tengö, Brondizio, Elmqvist, Malmer and Spierenburg2014). Different types of expertise are considered, and experts can be academics, but can also be civil servants, working in civil society organizations and the private sector. All experts have to inform IPBES of any potential conflicts of interests.

In addition, responding to calls to integrate diverse knowledge systems, and in particular, the knowledge of IPLCs (Fazey et al., Reference Fazey, Schäpke, Caniglia, Hodgson, Kendrick, Lyon, Page, Patterson, Riedy, Strasser, Verveen, Adams, Goldstein, Klaes, Leicester, Linyard, McCurdy, Ryan, Sharpe and Young2020; Raymond et al., Reference Raymond, Fazey, Reed, Stringer, Robinson and Evely2010; Turnhout et al., Reference Turnhout, Bloomfield, Hulme, Vogel and Wynne2012), IPBES has developed strategies to incorporate knowledge from diverse disciplinary perspectives, languages, knowledge systems, and regions (Montana & Borie, Reference Montana and Borie2016; Stokland et al., Reference Stokland, Stenseke and Emery2022). Examples of this are the explicit consideration of IPLCs worldviews in the IPBES conceptual framework, the institution of a taskforce on Indigenous and local knowledge (ILK) providing guidance and support for the integration of ILK across IPBES deliverables, the establishment of a formal approach for including ILK in IPBES procedures, the conformation of ILK liaison groups within assessment teams, and the delivery of ILK dialogues formally established within the assessment production process (Díaz et al., Reference Díaz, Demissew, Carabias, Joly, Lonsdale, Ash, Larigauderie, Adhikari, Arico, Báldi, Bartuska, Baste, Bilgin, Brondizio, Chan, Figueroa, Duraiappah, Fischer, Hill and Zlatanova2015; Hill et al., Reference Hill, Adem, Alangui, Molnár, Aumeeruddy-Thomas, Bridgewater, Tengö, Thaman, Adou Yao, Berkes, Carino, Carneiro da Cunha, Diaw, Díaz, Figueroa, Fisher, Hardison, Ichikawa, Kariuki and Xue2020; McElwee et al., Reference McElwee, Fernández‐Llamazares, Aumeeruddy‐Thomas, Babai, Bates, Galvin, Guèze, Liu, Molnár, Ngo, Reyes‐García, Roy Chowdhury, Samakov, Shrestha, Díaz and Brondízio2020). Through these processes, ILK holders and experts as well as non-governmental conservation organizations are invited to participate in all IPBES processes (as observers and experts) to ensure epistemic pluralism (Krug et al., Reference Krug, Sterling, Cadman, Geschke, Drummond de Castro, Schliep, Osemwegie, Muller-Karge and Maraseni2020; Stevance et al., Reference Stevance, Bridgewater, Louafi, King, Beard, Van Jaarsveld, Ofir, Kohsaka, Jenderedijan, Rosales Benites, Mulongoy, Chaudhari, Painter and Meter2020). Despite these efforts, we acknowledge that achieving the inclusion of diverse IPLC voices remains a challenge. Particularly, more powerful or vocal IPLCs may be overrepresented in IPBES spaces and assessments. Continuous and innovative efforts need to be developed to better represent and involve plural worldviews in IPBES and more generally in ISPPs.

Following IPBES protocols, the elaboration of assessments follows three phases: (i) the request and production of a scoping document delineating the scope of the assessment, (ii) the production of the assessment through expert evaluation of the state of knowledge and two open external reviews, and (iii) the approval of the summary for policymakers of the assessment and the acceptance of the chapters providing the supporting evidence (IPBES, 2018). Each of these phases is characterized by principles of inclusivity (of governments and stakeholders), transparency (through independent peer-review processes), and representation of both findings and gaps in a policy-relevant manner.

Yet, key challenges remain and have been noted in a range of studies about IPBES. First, equal representation of experts from all five UN regions is rarely achieved (Báldi & Palotás, Reference Báldi and Palotás2021; McElwee et al., Reference McElwee, Fernández‐Llamazares, Aumeeruddy‐Thomas, Babai, Bates, Galvin, Guèze, Liu, Molnár, Ngo, Reyes‐García, Roy Chowdhury, Samakov, Shrestha, Díaz and Brondízio2020; Montana & Borie, Reference Montana and Borie2016; Timpte et al., Reference Timpte, Montana, Reuter, Borie and Apkes2018), which can reduce the diversity of perspectives, priorities, and experiences that are key inputs for the assessments (Obura et al., Reference Obura, Katerere, Mayet, Kaelo, Msweli, Mather and Nantongo2021), and of the languages used in the selection of literature to be assessed (Lynch et al., Reference Lynch, Fernández-Llamazares, Palomo, Jaureguiberry, Amano, Basher, Lim, Mwampamba, Samakov and Selomane2021). Epistemic bias is also of concern. Recent studies identify a disciplinary bias, arguing that the social sciences are still not sufficiently represented in IPBES assessments (Díaz-Reviriego et al., Reference Díaz-Reviriego, Turnhout and Beck2019; Stokland et al., Reference Stokland, Stenseke and Emery2022; Vadrot et al., Reference Vadrot, Jetzkowitz and Stringer2016). In addition, the absence of guidelines on how to assess the validity of non-scientific work can explain the difficulty of integrating diverse knowledge systems, and in particular ILK, to the platform's work (Gustafsson et al., Reference Gustafsson, Berg, Lidskog and Löfmarck2019).

1.4 Case study and objectives of the research

The IPBES Methodological Assessment of the Diverse Values and Valuation of Nature (hereafter ‘Values Assessment’, VA) (IPBES, 2022a) and its summary for policymakers (IPBES, Reference Pascual, Balvanera, Christie, Baptiste, González-Jiménez, Anderson and Vatn2022b) were approved by the IPBES plenary in July 2022. The VA explores to what extent, and in which ways, diverse values of nature have been, and can better be, incorporated into decision-making to promote transformative changes to more just and sustainable futures. It also sheds light on the implications of including or excluding different types of values of nature from decision-making processes. Given the mandate of the VA to address values and valuation of nature from a diverse and inclusive perspective, the production of the VA was accompanied by unprecedented efforts to balance robustness and epistemic plurality within the assessment (Balvanera et al., Reference Balvanera, Pascual, Christie, Baptiste, Lliso, Monroy, Guibrunet, Anderson, Athayde, Barton, Chaplin-Kramer, Jacobs, Kelemen, Kumar, Lazos, Martin, Mwampamba, Nakangu, O'Farrell, González-Jiménez, Balvanera, Pascual, Christie, Baptiste and González-Jiménez2022). Here, we assess how effective IPBES efforts have been toward achieving plurality in the case of the VA, and reflect, as experts directly engaged in producing the VA, what could improve in future IPBES and other ISPP assessments. We aim to respond to the following question: To what extent is the linguistic and disciplinary diversity of experts involved in the VA reflected in a geographic and epistemic pluralism of the literature reviewed in the VA?

To address this general question, we answer these specific questions:

1. (1) How diverse was the team of authors in terms of their disciplinary background and geographical location?

2. (2) What strategies were used to identify plural literature?

3. (3) Is the literature cited in the VA plural from a geographic and epistemic standpoint?

4. (4) Which initiatives best contributed to enhance pluralism of the assessed literature?

The paper is structured as follows. We first present the mixed-methods approach involving a bibliometric analysis. Then, we present the results, and highlight various strategies that can help enhance geographic and epistemic pluralism in ISPP assessments.

2.1 Positionality and methodological approach

All authors of this paper participated in the IPBES VA in different capacities: PB, MC, BP, and UP were co-chairs of the assessment, with the role of leading the assessment and coordinating the experts who produced the assessment, as well as being coordinating lead authors of Chapter 1 of the assessment. GA-R, MC-F, VC, DG-J, and LG formed part of the technical support unit, with a technical and administrative role supporting the experts. ASM-S and BL were authors of Chapter 1. THM was author of Chapter 1 and coordinating lead author of Chapter 3. DFM and AVOC were contributing authors to Chapter 1. We used our direct involvement in the production of the VA to reflect on the different initiatives engaged to achieve plural literature reviews.

We used a mixed-methods approach to address the research questions. First, we built on existing descriptions of the VA process (Balvanera et al., Reference Balvanera, Pascual, Christie, Baptiste, Lliso, Monroy, Guibrunet, Anderson, Athayde, Barton, Chaplin-Kramer, Jacobs, Kelemen, Kumar, Lazos, Martin, Mwampamba, Nakangu, O'Farrell, González-Jiménez, Balvanera, Pascual, Christie, Baptiste and González-Jiménez2022; Pascual et al., Reference Pascual, Balvanera, Anderson, Chaplin-Kramer, Christie, González-Jiménez, Martin, Raymond, Termansen, Vatn, Athayde, Baptiste, Barton, Jacobs, Kelemen, Kumar, Lazos, Mwampamba, Nakangu and Zent2023) to describe the diversity of the expert team and of the literature review strategies. We produced descriptive statistics about the composition of the expert team, including the languages they speak, their nationality, and their disciplinary backgrounds. We grouped nationalities according to the UN regional groups of member states (United Nations, n.d.). We grouped literature review strategies as structured, semi-structured, and non-structured (see Pascual et al., Reference Pascual, Balvanera, Anderson, Chaplin-Kramer, Christie, González-Jiménez, Martin, Raymond, Termansen, Vatn, Athayde, Baptiste, Barton, Jacobs, Kelemen, Kumar, Lazos, Mwampamba, Nakangu and Zent2023). Second, we performed a bibliometric analysis to assess the pluralism of the literature cited in the VA; this is described in more detail below. Additionally, we collectively reflected on our experience as authors of the VA to identify successful strategies to achieve geographic and epistemic pluralism.

2.2 Bibliometric analysis

We conducted a bibliometric analysis on a random sample of documents cited in the VA (documents are defined as any piece of written text, as well as pictures, videos and other digital sources cited in the VA chapters' bibliography or reviewed as part of the VA's literature reviews). (We started our analysis in 2020. Given this, we only included in our analysis the documents cited in the Values Assessment's Second Order Draft, which was finalized in 2020. The Second Order Draft is a draft published 1 or 2 years before publication of the final version of the document, and it is when new sources stop being included in IPBES assessments. Therefore, very few additional sources would appear in the final assessment (only in the case of ‘outstanding importance’ published during that year) (IPBES, 2018).) We used Yamane's formula to calculate the sample size (i.e. number of documents analyzed), opting for a 97.5% confidence level. Yamane's formula is as follows (Yamane, Reference Yamane1967):

$$n = \displaystyle{N \over {1 + N{( e ) }^2}}$$

where n is the sample size, N is the population size (in our case, the 13,067 documents cited in the assessment), e is the level of precision (in our case, 0.025 for a 97.5% confidence level).

The sample size was calculated as follows:

$$n = \displaystyle{{13, 067} \over {1 + 13, 067{( {0.025} ) }^2}} = 1425$$

We randomly selected documents stratifying across chapters to ensure that at least 10% of sources from each chapter were included in the analysis (see Table 1). After deleting duplicates (i.e. documents cited in more than one chapter), our final sample contained 1682 documents, surpassing the number required to achieve a 97.5% confidence level.

Table 1.

Number of documents included in the bibliometric analysis

^a Chapter 1 – The role of the values of nature and valuation for addressing the biodiversity crisis and navigating toward more just and sustainable futures; Chapter 2 – Conceptualizing the diverse values of nature and their contributions to people; Chapter 3 – The potential of valuation; Chapter 4 – Value expression in decision-making; Chapter 5 – The role of diverse values of nature in visioning and transforming toward just and sustainable futures; Chapter 6 – Policy options and capacity development to operationalize the inclusion of diverse values in decision-making.

We selected different variables to assess the geographic and epistemic knowledge pluralism of the documents cited in the assessment. To assess geographic pluralism (defined as the integration of knowledge that is produced in diverse socio-ecological contexts), we used four variables. First, we identified the region of focus of the document, understood as the socio-ecosystem analyzed in the document. Ideally, the knowledge on a given region would be produced by authors within that same region, to reflect local knowledge and diverse perspectives on the matter at hand. Therefore, the second variable is the country where the first author resides and works (grouped according to UN political regions), and the third is whether another author resides and works in a different region than that of the first author. This allowed us to identify the regions which mostly finance and produce the documents cited in the assessment, and to identify works which may reflect a collaboration across geopolitical contexts. Finally, the fourth variable is the language in which the work was published. This allowed us to reflect on how language pluralism interacts with geographic and epistemic pluralism.

To understand the extent of epistemic pluralism (defined as the integration of knowledge from different theoretical perspectives and knowledge systems) in the VA, we used two different approaches. The first one, focusing on the different knowledge systems, was assessed using two variables: (i) the sector to which the author belongs (academic, governmental, IPLCs, or other, thus highlighting the arena in which the knowledge was developed); and (ii) the type of document, whether a book, a report, a law, a piece of artwork, etc. (assuming that different types of documents reflect different ways in which knowledge is shared). The second approach was to assess the academic discipline of the document (as a proxy of the different theoretical perspectives framing those documents); yet, this variable was only available for articles published in journals indexed in Scopus (which represent 54% of all the documents analyzed). The results associated with this variable must thus be interpreted with caution. As a result, our analysis of epistemic pluralism is centered around the diversity of knowledge systems.

Table 2 presents the variables used to characterize the sampled sources of the literature. The last column shows the percentage of documents for which data were available for each variable used in the bibliometric analysis. For instance, we could determine the region of origin for 95% of documents (i.e. for the remaining 5%, we were unable to identify the country of affiliation of the first author).

Table 2.

Variables included in the bibliometric analysis

^a This group is composed of the following countries, besides Western European countries: Australia, Israel, Canada, Turkey, United States of America, and New Zealand.

* The graphs presented in Section 3 present results based on the analysis of the subset of documents for which the data was available.

** The remaining 45% of documents did not focus on a particular region, for instance, they were theoretical papers.

3.1 Efforts to achieve geographic and epistemic pluralism in the VA

The assessment comprised of six chapters prepared over the course of 4 years (2018–2022) by 84 experts nominated by member governments or IPBES stakeholders, and selected by the Multidisciplinary Expert Panel of IPBES, and 11 review editors approved by the Management Committee of the Values Assessment. The expert team was balanced in terms of gender and disciplinary background: 54% of experts identified as women; 54% of experts held at least one higher education degree in the social sciences, and 54% held at least one higher education degree in biophysical sciences. Two-thirds held higher education degrees in at least two different disciplines, illustrating the multidisciplinary background of the expert team. Experts were citizens of 47 countries and spoke 51 languages. Eighteen authors identified as experts on IPLCs, and two as ILK holders. (‘Indigenous and local knowledge holders are understood to be persons situated in the collective knowledge systems of Indigenous peoples and local communities with knowledge from their own Indigenous peoples and local communities’; experts on Indigenous peoples and local communities are ‘persons who have knowledge about indigenous and local knowledge and associated issues, not necessarily from IPLCs’ (IPBES, 2023).)

The diversity of the expert team was enhanced by the participation of over 200 contributing authors (who were citizens of 49 different countries, and included 25 ILK experts and 12 ILK holders). Contributing authors were integrated into the different chapters to address specific literature gaps outside the authors' expertise, whether in terms of topics, non-scientific documents, or evidence from specific regions. Sixty-nine percent of experts were academics hired at universities, the rest were employed by public or private research institutes (18%), foundations, non-governmental organizations, or cooperation agencies (7%), independent researchers (4%), or civil servants (2%).

In the assessment, different strategies were used to identify and review the literature to be assessed: 29 literature reviews were conducted, of which five were structured, 10 were semi-structured, two were non-structured, two were invited contributions, and 10 were a combination of reviews. These have been defined as follows:

An analysis of the diversity yielded through each of these reviews is provided in Section 3.3. Ten of the literature reviews included languages other than English, and 15 included gray literature (Pascual et al., Reference Pascual, Balvanera, Anderson, Chaplin-Kramer, Christie, González-Jiménez, Martin, Raymond, Termansen, Vatn, Athayde, Baptiste, Barton, Jacobs, Kelemen, Kumar, Lazos, Mwampamba, Nakangu and Zent2023). Combining expert knowledge and unstructured approaches to identify the literature, with structured protocols to review the literature identified, allowed experts speaking languages other than English to integrate literature in their native languages to the reviews and share key findings with the rest of experts in a systematized manner.

One crucial strategy as part of the literature reviews was the use of keywords beyond those used in the IPBES conceptual framework. The IPBES conceptual framework builds on a limited set of concepts (which include both scientific terms and non-scientific terms, such as ‘Mother Earth’) to be useful as a boundary object to communicate across cultures on biodiversity and its relations with human wellbeing (Díaz et al., Reference Díaz, Demissew, Carabias, Joly, Lonsdale, Ash, Larigauderie, Adhikari, Arico, Báldi, Bartuska, Baste, Bilgin, Brondizio, Chan, Figueroa, Duraiappah, Fischer, Hill and Zlatanova2015). Complementing keyword searches based on the conceptual framework by other non-scientific and regional terms was useful in achieving geographic and epistemic pluralism. For example, a literature search conducted around the concept of ‘Good Living’ allowed to integrate concepts such as ‘Buen Vivir’ from South America but also more than 20 other related concepts such as ‘Mauri Ora’, ‘Alli Kawsay’, ‘Minobimatisiwin’, ‘Vida sabrosa’, ‘Ubuntu’, ‘Satoumi’, among others, to refer to a philosophy of living in harmony with nature from different Indigenous and local perspectives. Using these keywords enabled the identification of literature in other languages such as Spanish, Portuguese, and French, and from the social sciences, humanities, and Indigenous local knowledge, providing alternative perspectives on the diverse values of nature. The additional keywords (those that did not emerge directly from the IPBES conceptual framework) were identified through the knowledge of VA authors and of ILK experts and holders who participated in the ILK dialogues, as well as through a snowball strategy (keywords were identified by reading existing literature on diverse values of nature). This allowed the use of terms that were not originally identified into literature searches.

To enhance legitimacy, relevance, and credibility, the assessment was produced in a stepwise process consisting of five drafts, all of which were reviewed through an internal peer-review process. Two drafts – the First Order Draft and the Second Order Draft – were also reviewed externally by non-IPBES experts and stakeholders and IPBES member governments to produce the final assessment report. Throughout the external review processes, stakeholders and member governments could identify gaps in the assessment and suggest additional literature to include in the reviews. All comments received through the review had to be addressed by experts, either by incorporating the comment into the VA or by offering a justification for why they did not. This process was crucial in finding and addressing research gaps and biases. As an example, the First Order Draft received 2836 comments from 210 expert reviewers and 11 governments.

In addition, different efforts were undertaken to ensure that ILK was integrated across the VA assessment. Some of these efforts are already part of the IPBES rules of procedure for producing assessments and are led by an ILK Taskforce and Technical Support Unit. An ILK liaison group (with assessment authors) was formed to ensure that ILK was considered throughout the assessment. Three ILK Dialogues were held with the aim of exchanging experiences, opinions, and worldviews regarding ‘values of nature’ from different IPLCs. These dialogues had around 160 participants representing more than 50 countries. Another effort was the Global Call for ILK contributions, launched in six languages (Arabic, English, French, Russian, Chinese, and Spanish), which aimed to gather and incorporate evidence from ILK holders. Through this Global Call, authors received 420 contributions from 73 countries; 33% of contributors self-identified as indigenous peoples and 22% identified as members of local communities. (Other contributors did not self-identify as such. This means that they can be experts on Indigenous issues without self-identifying as being a member of an Indigenous or local community.) We recognize that participation in the ILK dialogues required the capacity to travel. Likewise, the participation in the Global Call for ILK contributions required knowledge of one of the six UN official languages. This may have hindered the participation of some IPLCs. These tools can enhance pluralism in the assessment processes but cannot ensure a complete representation of all IPLCs. Additional efforts led by the ILK liaison groups included: (1) building collaborative review efforts focused on ILK, building on experts' interdisciplinary background and multilingualism to shape review protocols with ILK-specific search terms and in languages other than English to reach ILK sources; and (2) inviting contributing authors who were ILK holders from regions or countries where knowledge gaps were identified. Furthermore, these efforts also influenced the rest of the reviews by, for example, conducting semi-structured ILK reviews or providing ILK principles to be integrated in structured review protocols.

3.2 Pluralism achieved in the inclusion of the literature

The Assessment cited 13,067 sources of evidence (thereafter ‘documents’) that were reviewed in depth, including peer-reviewed academic literature, policy documents, and other sources such as ILK, regarding the diverse values of nature and their role in supporting decision-making processes. The results of the bibliometric analysis (Figures 1 and 2) illustrate the pluralism achieved in terms of geographic and epistemic diversity of literature in the VA.

Figure 1.

Geographic and linguistic diversity. Panel (a) shows the country of affiliation of the first author of the documents reviewed. Panel (b) shows the focus of the documents reviewed. Panel (c) shows the percentage of documents written by a cross-regional team (when authors come from more than one of the UN regions). Panel (d) shows the language in which documents were published.

Figure 2.

Diversity of disciplines and knowledge systems. Panel (a) describes the types of sources (academic, intergovernmental, non-intergovernmental, governmental, and other incl. indigenous contributions). Panel (b) shows the percentage of reviewed documents based on their format (journal article, report, book, review, book chapter, website, and other). Panel (c) shows the percentage of journal articles reviewed by discipline.

Although the expert team came from all of the five UN regions, three-quarters of documents included in the VA were produced by a first author based in the Global North, namely Western Europe and Other States (WEOS) (Figure 1a). (Authors who work in WEOS may be nationals of countries from other regions, and/or have been trained in other regions, as a result of the global mobility of scientists (Verginer & Riccaboni, Reference Verginer and Riccaboni2021). Yet, there is evidence that the research focus of researchers varies across world regions and tends to be related to local challenges and priorities (Salvia et al., Reference Salvia, Leal Filho, Brandli and Griebeler2019). Researchers' place of works therefore plays a role in the focus of their research. Hence, authors' professional affiliation is a useful, albeit partial, proxy of geographic pluralism.) Twenty-eight percent of documents were produced by multi-regional teams, which suggests that few documents with first authors based in WEOS have been written in collaboration with authors from other regions (Figure 1c). Of all the sources cited, 56% had an explicit geographic focus, and this focus was distributed across the five UN regions. Still, the WEOS region appears as the one with the greatest focus (77% of documents with a geographic focus, Figure 1b). Additionally, the predominant language of documents was English (96%), followed by Spanish (2%); the remaining sources were in Chinese, Dutch, French, German, Portuguese, Norwegian, Polish, Russian, and Ukrainian, or were published in multiple languages (Figure 1d). Most of the languages spoken by experts (for instance, Hindi, Afrikaans, Jotï, Korean, Farsi, Turkish, Obijwe, or Swahili) were not represented in the literature reviewed in the assessment.

In terms of epistemic pluralism, we noted a primacy of academic documents (90%). Indigenous contributions represented just under 1% of documents, which amounts to 118 documents included in the assessment (Figure 2a). More than three-quarters of the sources used were journal articles. Books and reports were less common. Other documents, such as laws and public policies, videos, and artwork, represent 2% (n = 280) of the documents (Figure 2b). The predominant discipline of the journal articles was environmental sciences (50%), followed by social sciences (12%) and economics, econometrics, and finance (10%) (Figure 2c). Yet, this is not enough to ascertain a disciplinary bias in the assessment. Indeed, environmental sciences journals may publish articles from the social sciences or that combine social and natural sciences. Additionally, this disciplinary analysis only focused on 54% of the total of documents included in our corpus of assessed literature. Articles published in journals not indexed in Scopus, as well as books and book chapters, were not included. Incidentally, these are media that are relatively more favored by social scientists to publish their results. Therefore, our analysis remains inconclusive as to the disciplinary pluralism achieved as part of the VA.

3.3 Diverse literature review strategies can enhance pluralism

Differences were found in the types of documents identified by different literature reviews (Figure 3). Structured and semi-structured literature reviews were found to be more effective in identifying academic sources. Unstructured reviews identified a greater diversity of documents, especially related to Africa, and Latin America and the Caribbean (both written by a first author based in those regions and focusing on those regions).

Figure 3.

Pluralism achieved by type of literature review. It shows the diversity represented in the type of sources, languages, country of affiliation of the first author, and regions of focus across the analyzed literature; against six different review categories ranging from more structured reviews to expert selection of the literature.

Invited contributions, on the other hand (which include, for instance, the ILK global call for contributions), provided a great extent of non-academic documents, particularly indigenous and governmental documents. They also provided a broad range of documents in languages other than English and written by a first author outside of Western Europe. For instance, 12% of documents identified through invited contributions focused on Latin America (one of the least represented regions in terms of focus, with only 5% of documents overall).

A significant number of documents (44%) were selected based on experts' knowledge (those that appear in Figure 3 as ‘main text’). Although those are mostly academic articles, they tend to be the most regionally diverse. For instance, 39% of documents included in the assessment that were written by a first author based in Eastern Europe (one of the least represented regions in terms of first authors' affiliation with only 1% of documents overall) were identified through expert knowledge, including both the assessment authors and contributing authors.