2.1.1. Forest extent and forest type mapping

Tracking the extent of forests is crucial to understand the spatial distribution of forest resources, ecosystem services, and assess the role of forest in land surface dynamics (Keenan et al., Reference Keenan, Reams, Achard, de Freitas, Grainger and Lindquist2015). Thereby, forest can be classified to different management, functional, or ecosystems types (e.g., coniferous, deciduous forests) (Buchhorn et al., Reference Buchhorn, Lesiv, Tsendbazar, Herold, Bertels and Smets2020; Zhang et al., Reference Zhang, Liu, Chen, Gao, Xie and Mi2021). In this regard, Earth observation data from long-term satellite missions (e.g., Landsat or Sentinel described in Section 4.4) enable to track forest extent dynamics across past decades (Hansen et al., Reference Hansen, Potapov, Moore, Hancher, Turubanova, Tyukavina, Thau, Stehman, Goetz, Loveland, Kommareddy, Egorov, Chini, Justice and Townshend2013), enabling to assess conservation efforts and anthropogenic land cover change, such as deforestation for agricultural expansion (Curtis et al., Reference Curtis, Slay, Harris, Tyukavina and Hansen2018).

2.1.2. Tree species mapping

A fine-scaled representation of forest stands in terms of their species composition is relevant for forestry (e.g., species-specific timber supply), biogeographical assessments (e.g., climate change-induced shifts in species distributions), or biodiversity monitoring (Fassnacht et al., Reference Fassnacht, Latifi, Stereńczak, Modzelewska, Lefsky, Waser, Straub and Ghosh2016; Wang and Gamon, Reference Wang and Gamon2019; Cavender-Bares et al., Reference Cavender-Bares, Gamon and Townsend2020). Recent developments in machine learning greatly advanced the identification of tree species in high-resolution data (e.g., imagery or LiDAR point clouds from drones and airplanes) using semantic and instance segmentation methods (Schiefer et al., Reference Schiefer, Kattenborn, Frick, Frey, Schall, Koch and Schmidtlein2020; Li et al., Reference Li, Chai, Wang, Lei and Zhang2022a; Cloutier et al., Reference Cloutier, Germain and Laliberté2023). At large spatial scales, Earth observation satellite data, providing coarser spatial but high temporal and spectral resolutions, enable accurate assessments of tree species distributions using spatiotemporal machine learning methods (Ienco et al., Reference Ienco, Interdonato, Gaetano and Minh2019; Bolyn et al., Reference Bolyn, Lejeune, Michez and Latte2022).

2.1.3. Biomass quantification

Forests provide cardinal ecosystem services through their provision of timber and their role as sinks in the terrestrial carbon cycle (Regnier et al., Reference Regnier, Resplandy, Najjar and Ciais2022). Tree biomass is primarily a product of the wood volume and density, while both properties are challenging to obtain from remote sensing data. Accurate biomass estimates of individuals trees can be obtained from close-range 3D representations acquired from terrestrial or drone-based LiDAR systems (Brede et al., Reference Brede, Terryn, Barbier, Bartholomeus, Bartolo, Calders, Derroire, Moorthy, Lau, Levick, Raumonen, Verbeeck, Wang, Whiteside, van der Zee and Herold2022). More indirectly related information on crown and canopy structure derived from airborne or spaceborne LiDAR and SAR data can be used to estimate biomass at the stand scale (Le Toan et al., Reference Le Toan, Quegan, Davidson, Balzter, Paillou, Papathanassiou, Plummer, Rocca, Saatchi, Shugart and Ulander2011; Lu et al., Reference Lu, Chen, Wang, Liu, Li and Moran2016). Some studies have indicated the value of passive optical data from satellites, since forest biomass is partially correlated with foliage density (Besnard et al., Reference Besnard, Koirala, Santoro, Weber, Nelson, Gütter, Herault, Kassi, N’Guessan, Neigh, Poulter, Zhang and Carvalhais2021; Potapov et al., Reference Potapov, Li, Hernandez-Serna, Tyukavina, Hansen, Kommareddy, Pickens, Turubanova, Tang, Silva, Armston, Dubayah, Bryan Blair and Hofton2021). Given that precise large-scale biomass distributions cannot be directly revealed through a single remote sensing modality alone, deep learning may play a crucial role to simultaneously exploit the suite and high dimensionality of available data modalities (Yang et al., Reference Yang, Liang and Zhang2020).

2.1.4. Forest health, disturbance, and mortality

In many regions, forest ecosystems are under pressure as globalization facilitates the introduction of exotic pests and pathogens, climate change exceeds the resilience and resistance of trees (Hartmann et al., Reference Hartmann, Bastos, Das, Esquivel-Muelbert, Hammond, Martínez-Vilalta, McDowell, Powers, Pugh, Ruthrof and Allen2022), while nutrient and water cycles are affected by anthropogenic activities (Steffen et al., Reference Steffen, Sanderson, Tyson, Jäger, Matson, Moore, Oldfield, Richardson, Schellnhuber, Turner and Wasson2005; Trumbore et al., Reference Trumbore, Brando and Hartmann2015). A decline in tree health, for example, due to pathogen infestations or shortages of water and nutrients, can lead to a variety of symptoms, such as changing concentrations of multiple biochemical tissue properties (e.g., pigments, carbohydrates, and water content), which in turn can be sensed through multispectral or hyperspectral reflectance (Zarco-Tejada et al., Reference Zarco-Tejada, Camino, Beck, Calderon, Hornero, Hernández-Clemente, Kattenborn, Montes-Borrego, Susca, Morelli, Gonzalez-Dugo, PRJ, Landa, Boscia, Saponari and Navas-Cortes2018; Zarco-Tejada et al., Reference Zarco-Tejada, Hornero, Beck, Kattenborn, Kempeneers and Hernández-Clemente2019). In this context, deep learning algorithms are very promising, due to their capability to exploit high-dimensional data (e.g., hyperspectral) and to translate it into a suite of foliage properties relevant to vegetation health (Cherif et al., Reference Cherif, Feilhauer, Berger, Dao, Ewald, Hank, He, Kovach, Lu, Townsend and Kattenborn2023). An interconnected topic are globally increased rates of forest mortality (Allen et al., Reference Allen, Macalady, Chenchouni, Bachelet, McDowell, Vennetier, Kitzberger, Rigling, Breshears, Hogg, Gonzalez, Fensham, Zhang, Castro, Demidova, Lim, Allard, Running, Semerci and Cobb2010; Hartmann et al., Reference Hartmann, Bastos, Das, Esquivel-Muelbert, Hammond, Martínez-Vilalta, McDowell, Powers, Pugh, Ruthrof and Allen2022). In this context, a wealth of approaches was successfully employed at local scales, such as the detection of dead trees via semantic or instance segmentation techniques (Sani-Mohammed et al., Reference Sani-Mohammed, Yao and Heurich2022; Cloutier et al., Reference Cloutier, Germain and Laliberté2023), or at large scales, such as the regression of annual cover of dead tree crowns in satellite image pixels with deep learning-based time series analysis (Schiefer et al., Reference Schiefer, Schmidtlein, Frick, Frey, Klinke, Zielewska-Büttner, Junttila, Uhl and Kattenborn2023).

2.1.5. Biophysical traits and functional ecosystem properties

With accelerated biodiversity decline and environmental change, understanding functional properties, their diversity across stands and landscapes, as well as their phenology (temporal dynamics), is essential to assess the resilience and resistance of ecosystems (Thompson et al., Reference Thompson, Mackey, McNulty and Mosseler2009; Sakschewski et al., Reference Sakschewski, Von Bloh, Boit, Poorter, Peña Claros, Heinke, Joshi and Thonicke2016). Given that trees through evolution developed different strategies to interact with light, their appearance studied with optical remote sensing signals can inform on a variety of functional traits, such as the foliage density, date of green up, or contents of different pigments, and carbohydrates (Schneider et al., Reference Schneider, Morsdorf, Schmid, Petchey, Hueni, Schimel and Schaepman2017; Cherif et al., Reference Cherif, Feilhauer, Berger, Dao, Ewald, Hank, He, Kovach, Lu, Townsend and Kattenborn2023). Such functional traits determine the configuration of an ecosystem and thereby modulate functional ecosystem processes (Migliavacca et al., Reference Migliavacca, Musavi, Mahecha, Nelson, Knauer, Baldocchi, Perez-Priego, Christiansen, Peters, Anderson, Bahn, Andrew Black, Blanken, Bonal, Buchmann, Caldararu, Carrara, Carvalhais, Cescatti, Chen, Cleverly, Cremonese, Desai, El-Madany, Farella, Fernández-Martínez, Filippa, Forkel, Galvagno, Gomarasca, Gough, Göckede, Ibrom, Ikawa, Janssens, Jung, Kattge, Keenan, Knohl, Kobayashi, Kraemer, Law, Liddell, Ma, Mammarella, Martini, Macfarlane, Matteucci, Montagnani, Pabon-Moreno, Panigada, Papale, Pendall, Penuelas, Phillips, Reich, Rossini, Rotenberg, Scott, Stahl, Weber, Wohlfahrt, Wolf, Wright, Yakir, Zaehle and Reichstein2021; Gomarasca et al., Reference Gomarasca, Migliavacca, Kattge, Nelson, Niinemets, Wirth, Cescatti, Bahn, Nair, Acosta, Altaf Arain, Beloiu, Andrew Black, Bruun, Bucher, Buchmann, Byun, Carrara, Conte, da Silva, Duveiller, Fares, Ibrom, Knohl, Komac, Limousin, Lusk, Mahecha, Martini, Minden, Montagnani, Mori, Onoda, Peñuelas, Perez-Priego, Poschlod, Powell, Reich, Šigut, van Bodegom, Walther, Wohlfahrt, Wright and Reichstein2023), that is, fluxes of energy and matter between the terrestrial biosphere, pedosphere, hydrosphere, and atmosphere, including carbon, evapotranspiration, latent, and sensible heat. Due to the cardinal importance of these fluxes in the Earth system, considerable efforts have been made to monitor them using a ground-based sensor network of flux towers (e.g., FLUXNET) (Baldocchi et al., Reference Baldocchi, Falge, Gu, Olson, Hollinger, Running, Anthoni, Bernhofer, Davis, Evans, Fuentes, Goldstein, Katul, Law, Lee, Malhi, Meyers, Munger, Oechel, Paw U, Pilegaard, Schmid, Valentini, Verma, Vesala, Wilson and Wofsy2001). Given the complexity of these ecosystem processes, deep learning is assumed to greatly enlarge our capabilities to exploit local flux towers and globally available remote sensing data to spatially and temporally extrapolate and understand forest ecosystem process (Jung et al., Reference Jung, Koirala, Weber, Ichii, Gans, Camps-Valls, Papale, Schwalm, Tramontana and Reichstein2019; Reichstein et al., Reference Reichstein, Camps-Valls, Stevens, Jung, Denzler, Carvalhais and Prabhat2019; Camps-Valls et al., Reference Camps-Valls, Tuia, Zhu and Reichstein2021; ElGhawi et al., Reference ElGhawi, Kraft, Reimers, Reichstein, Körner, Gentine and WinklerWinkler2023).

2.2.1. Tree species

There are an estimated 73,000 tree species on Earth (Cazzolla Gatti et al., Reference Cazzolla Gatti, Reich, Gamarra, Crowther, Hui, Morera, Bastin, de Miguel, Nabuurs, Svenning, Serra-Diaz, Merow, Enquist, Kamenetsky, Lee, Zhu, Fang, Jacobs, Pijanowski, Banerjee, Giaquinto, Alberti, Almeyda Zambrano, Alvarez-Davila, Araujo-Murakami, Avitabile, Aymard, Balazy, Baraloto, Barroso, Bastian, Birnbaum, Bitariho, Bogaert, Bongers, Bouriaud, Brancalion, Brearley, Broadbent, Bussotti, Castro Da Silva, César, Češljar, Chama Moscoso, Chen, Cienciala, Clark, Coomes, Dayanandan, Decuyper, Dee, Del Aguila Pasquel, Derroire, Djuikouo, Van Do, Dolezal, Đorđević, Engel, Fayle, Feldpausch, Fridman, Harris, Hemp, Hengeveld, Herault, Herold, Ibanez, Jagodzinski, Jaroszewicz, Jeffery, Johannsen, Jucker, Kangur, Karminov, Kartawinata, Kennard, Kepfer-Rojas, Keppel, Khan, Khare, Kileen, Kim, Korjus, Kumar, Kumar, Laarmann, Labrière, Lang, Lewis, Lukina, Maitner, Malhi, Marshall, Martynenko, Monteagudo Mendoza, Ontikov, Ortiz-Malavasi, Pallqui Camacho, Paquette, Park, Parthasarathy, Peri, Petronelli, Pfautsch, Phillips, Picard, Piotto, Poorter, Poulsen, Pretzsch, Ramírez-Angulo, Restrepo Correa, Rodeghiero, Rojas Gonzáles, Rolim, Rovero, Rutishauser, Saikia, Salas-Eljatib, Schepaschenko, Scherer-Lorenzen, Šebeň, Silveira, Slik, Sonké, Souza, Stereńczak, Svoboda, Taedoumg, Tchebakova, Terborgh, Tikhonova, Torres-Lezama, Van Der Plas, Vásquez, Viana, Vibrans, Vilanova, Vos, Wang, Westerlund, White, Wiser, Zawiła-Niedźwiecki, Zemagho, Zhu, Zo-Bi and Liang2022), the majority of which are found in the tropics. While tree species show many similarities (e.g., the presence of woody stems and branches), every tree species differs from one another in their chemical and structural make-up and how they will reflect solar radiation (Asner et al., Reference Asner, Martin, Carranza-Jiménez, Sinca, Tupayachi, Anderson and Martinez2014). For example, tree foliage of different species comes in various shades of green that reflect the concentrations of pigments (e.g., chlorophylls and carotenoids) (Gates et al., Reference Gates, Keegan, Schleter and Weidner1965). Likewise, tree species differ from each other in their leaf form crown structure (Verbeeck et al., Reference Verbeeck, Bauters, Jackson, Shenkin, Disney and Calders2019), which will affect the remotely sensed signal. Such foliar biophysical and crown structural variation among tree species is the result of millions of years of evolution and of adaptations to various environmental conditions (Meireles et al., Reference Meireles, Cavender-Bares, Townsend, Ustin, Gamon, Schweiger, Schaepman, Asner, Martin, Singh, Schrodt, Chlus and O’Meara2020).

From a machine learning perspective, the biggest challenge associated with tree species diversity is that models trained on data from a given set of tree species might transfer poorly to other regions that host different species. However, the phylogeny and evolutionary distances of tree species are fairly well known (Zanne et al., Reference Zanne, Tank, Cornwell, Eastman, Smith, FitzJohn, McGlinn, O’Meara, Moles, Reich, Royer, Soltis, Stevens, Westoby, Wright, Aarssen, Bertin, Calaminus, Govaerts, Hemmings, Leishman, Oleksyn, Soltis, Swenson, Warman and Beaulieu2014), and tree species that are closer phylogenetically tend to be more similar in their traits (Ackerly, Reference Ackerly2009). As such, phylogenetic correlations and distances among tree species can potentially be leveraged to improve model transferability. Another interconnected challenge, elaborated upon in the following sections, pertains to the dynamic nature of tree species in relation to their leaf biophysical and crown structural characteristics. Instead, each individual differs according to their abiotic (e.g., microclimate and soil) and biotic environment (competition and herbivory), and as such, the expression of foliage and crown properties can overlap between species (Fassnacht et al., Reference Fassnacht, Latifi, Stereńczak, Modzelewska, Lefsky, Waser, Straub and Ghosh2016).

2.2.2. Seasons and phenology

Trees are sessile organisms, but they still respond dynamically to fluctuating seasons. In some cases, phenological properties, such as leaf onset, flowers, or seeds, might be of direct interest to monitor ecological phenomena or species (Wagner, Reference Wagner2021), in which case high-frequency multitemporal imagery might be required. Indeed, phenological changes among species, for example, changes in leaf color during autumn senescence, can help to distinguish tree species based on color, which can be used to improve species classification models (Cloutier et al., Reference Cloutier, Germain and Laliberté2023). However, phenological properties may also hinder the transferability of models across time (Kattenborn et al., Reference Kattenborn, Schiefer, Frey, Feilhauer, Mahecha and Dormann2022b). For instance, the information learnt by a machine learning model using data acquired in summer may not transfer to the same location in fall as trees may have changed in their leaf biochemical properties or the fraction of flowers and seeds in the canopy (Schiefer et al., Reference Schiefer, Schmidtlein and Kattenborn2021). In such cases, the temporal representativeness of data on individual tree species can be key (Kattenborn et al., Reference Kattenborn, Schiefer, Frey, Feilhauer, Mahecha and Dormann2022b).

2.2.3. Forest dynamics

The structure and composition of forests is strongly influenced by abiotic factors such as climate, geology, and soils, as well as water availability. For example, declining temperatures and/or growing season lengths with increasing latitude and/or elevation can filter out tree species that cannot tolerate low temperatures (e.g., low frost resistance) or that do not have enough time to produce mature tissue once the growing season becomes too short (Körner et al., Reference Körner, Basler, Hoch, Kollas, Lenz, Randin, Vitasse and Zimmermann2016). In addition, changes in soil nutrient availability driven by geomorphological processes can influence forest canopy biochemistry (Chadwick and Asner, Reference Chadwick and Asner2018). Water supply is also important: too much water favors trees that can tolerate waterlogging, whereas too little water favors trees that can resist or recover from xylem cavitation (Choat et al., Reference Choat, Jansen, Brodribb, Cochard, Delzon, Bhaskar, Bucci, Feild, Gleason, Hacke, Jacobsen, Lens, Maherali, Martínez-Vilalta, Mayr, Mencuccini, Mitchell, Nardini, Pittermann, Pratt, Sperry, Westoby, Wright and Zanne2012). Much of these environmental influences on forest composition express themselves via tree species turnover, that is, changes in tree species composition across these spatial environmental gradients or discontinuities. However, changes in forest composition and structure can also arise through intraspecific variation. Applications of machine learning methods to forest monitoring should integrate these sources of variation. In particular, incorporating environmental drivers of forest composition and structure as model inputs may help to transfer forest monitoring models from one region to the other.

Tree monitoring can also be affected by biotic factors—that is, by other organisms. First, pests and pathogens can impact tree health, foliage chemistry, and/or water content, which in turn can affect the remotely sensed signal of forest canopies (Sapes et al., Reference Sapes, Lapadat, Schweiger, Juzwik, Montgomery, Gholizadeh, Townsend, Gamon and Cavender-Bares2022). The health status of trees is often directly expressed via their foliage properties and crown architecture and therefore can cause a large variability in remote sensing signals (Zarco-Tejada et al., Reference Zarco-Tejada, Camino, Beck, Calderon, Hornero, Hernández-Clemente, Kattenborn, Montes-Borrego, Susca, Morelli, Gonzalez-Dugo, PRJ, Landa, Boscia, Saponari and Navas-Cortes2018; Kattenborn et al., Reference Kattenborn, Richter, Guimarães-Steinicke, Feilhauer and Wirth2022a). In addition, the remotely sensed signals of trees can also be “overshadowed” by other organisms that live in their crowns (epiphytes), particularly in tropical environments (Baldeck et al., Reference Baldeck, Asner, Martin, Anderson, Knapp, Kellner and Wright2015). Prominent examples are lianas or mistletoes.

Forest management activities as part of forest dynamics, such as harvesting, thinning, and pruning, can challenge the accurate mapping of forest attributes with remote sensing, as this crucial information is often unavailable but significantly impacts forest structure and composition. This lack of data therefore introduces uncertainty into remote sensing analyses and models.

2.2.4. Data collection

As previously mentioned, forests can exhibit significant diversity in terms of their composition and structure across different locations and time periods due to a variety of factors. This heterogeneity poses a particular difficulty in creating machine learning models for forest monitoring. Models developed for one region may not easily generalize to other regions that lie beyond the scope of the training data distribution. One solution for this issue would involve training these models using extensive datasets that encompass the complete spectrum of conditions present in diverse forest environments. Forest remote sensing data worldwide, especially acquired from sensors on satellites, are abundant and generally easily obtainable. In sharp contrast, there is a scarcity of ground-based data, including labels or annotations.

In contrast to other disciplines, annotating remote sensing data in the context of vegetation is often time-consuming, costly, and complex. This phenomenon arises due to the fact that vegetation of various species or conditions frequently exhibits striking similarities, often referred to as “greenery.” Moreover, vegetation communities often show smooth transitions across species or growth forms along environmental gradients. This aspect adds another layer of complexity to the task of distinguishing between individual plants, species, or growth forms (Kattenborn et al., Reference Kattenborn, Leitloff, Schiefer and Hinz2021). Often, field inventories become essential to validate annotations, such as the identification of tree species (Kattenborn et al., Reference Kattenborn, Eichel, Wiser, Burrows, Fassnacht and Schmidtlein2020; Cloutier et al., Reference Cloutier, Germain and Laliberté2023), or when the properties of interest, such as stem diameters, cannot be directly extracted from remote sensing data and require on-site measurements conducted by human observers. Gathering such field data is typically a time-intensive, expensive, and gradual process, leading to significant constraints on its accessibility. Field data such as stem diameters are very important to estimate aboveground tree biomass because most published allometric equations use stem diameter as its primary predictor (Gonzalez-Akre et al., Reference Gonzalez-Akre, Piponiot, Lepore, Herrmann, Lutz, Baltzer, Dick, Gilbert, He, Heym, Huerta, Jansen, Johnson, Knapp, Kral, Lin, Malhi, McMahon, Myers, Orwig, Rodriguez-Hernandez, Russo, Shue, Wang, Wolf, Yang, Davies and Anderson-Teixeira2022). Moreover, spatial coordinates frequently serve as the sole means of connecting field data to remote sensing data. However, GPS or GNSS geolocation in forest settings often introduces substantial uncertainties (ranging from meters to tens of meters), thereby posing challenges in accurately establishing a posteriori links between field observations and remote sensing data (Kattenborn et al., Reference Kattenborn, Leitloff, Schiefer and Hinz2021).

Therefore, integrating various datasets is a strategy aimed at addressing the scarcity of annotated data, cutting down annotation expenses, and lessening the dependency on field-based ground truthing. Nonetheless, this may result heterogenous datasets: In numerous cases, annotations vary (such as boxes, polygons, and points), as well as their quality, across different applications. Annotations are frequently customized to match particular remote sensing data characteristics, such as spatial resolution. Hence, directly merging labels from different datasets is often not feasible, or at the very least, alternative approaches are necessary. One such approach is weakly supervised learning, where the potential lack of label quality is counteracted by leveraging a substantial quantity of data (see Section 3.2.2).

The key takeaway from this section is that the development of machine learning models for forest monitoring will consistently involve a substantial surplus of unlabeled remote sensing data in comparison to labeled ground-truth data. This disparity arises due to the inherent challenges in obtaining labeled data. This scenario is not exclusive to forest monitoring; rather, it is a prevalent aspect shared with other geospatial applications using remote sensing data (Rahaman et al., Reference Rahaman, Weiss, Träuble, Locatello, Lacoste, Bengio, Pal, Li and Schölkopf2022; Mai et al., Reference Mai, Huang, Sun, Song, Mishra, Liu, Gao, Liu, Cong, Hu, Cundy, Li, Zhu and Lao2023a) (see Section 3.1.2). This has two main implications for machine learning research aimed at improving forest monitoring. First, there is a need to develop machine learning methods to forest monitoring that that can effectively utilize limited labeled data. One approach involves leveraging self-supervised learning techniques to extract valuable representations from the available data (see Section 3.2.1). Second, there exists a necessity for novel machine learning strategies, including active learning or alternative forms of model-assisted labeling. These approaches aim to expedite the process of label collection by human observers and reduce associated costs (see Section 3.2.3).

3.1.1. Domain adaptation

Transfer learning refers to transferring information learnt by a model on one set of problems to different set of problems (Weiss et al., Reference Weiss, Khoshgoftaar and Wang2016). For example, one may pretrain a model on a large, commonly used dataset and then fine-tune it on a smaller dataset representing the specific problem in question. Transfer learning can boost generalization when there is a significant distribution shift between training and inference (Csurka, Reference Csurka2017). One particularly notable approach to transfer learning is domain adaptation, where a model must be applied to target domains that are unknown or lacking labeled data (Soltani et al., Reference Soltani, Feilhauer, Duker and Kattenborn2022). Some domain adaptation approaches have already been applied in plant identification (Ganin and Lempitsky, Reference Ganin and Lempitsky2015). Autonomous driving has witnessed significant exploration in the realm of unsupervised domain adaptation (UDA), where a model is trained on labeled data from the source domain and unlabeled data from the target domain, with the objective of improving its performance specifically on the target domain. It has been explored in the context of unlabeled or unseen source or target domains (Wilson and Cook, Reference Wilson and Cook2020) using generative (Hoffman et al., Reference Hoffman, Tzeng, Park, Zhu, Isola, Saenko, Efros and Darrell2018) or adversarial (Vu et al., Reference Vu, Jain, Bucher, Cord and Pérez2019) methods. The UDA framework has also been explored for cross-modal learning considering domains from different sensor modalities (Jaritz et al., Reference Jaritz, Vu, de Charette, Wirbel and Pérez2020). Within the context of forest monitoring, this framework could prove particularly valuable for adapting a model from one forest to another, regardless of whether they belong to the same biome or not, to identify similar species across both the source and target domains (see Sections 2.1.2 and 2.2.1). Additionally, this framework would be beneficial for adapting the model to address distribution shifts that occur between tree signature distributions (see Sections 2.1.4, 2.2.2, and 2.2.3) as well as between different sensors (see Section 2.2.4). Domain adaptation has been investigated in the field of remote sensing mostly in the context of extrapolation across time or geographical region, including approaches for both aerial and satellite data (Shi et al., Reference Shi, Du, Guo and Du2022; Wang et al., Reference Wang, Feng, Sun, Zhang, Zhang, Yang and Meng2022; Arnaudo et al., Reference Arnaudo, Tavera, Masone, Dominici and Caputo2023; Ma et al., Reference Ma, Zhang, Wang and Pun2023; Xu et al., Reference Xu, Shi, Yuan and Zhu2023). Such work holds potential for training generalizable algorithms for forest monitoring, such as adapting models from PhenoCams to satellite images (Kosmala et al., Reference Kosmala, Hufkens and Richardson2018).

3.1.2. Foundation models

Foundation models are models that can operate on diverse sets of input modalities, scales, data regimes, and downstream tasks. They refer to large-scale multimodal and multitask models, which have opened up research in generalization capacities such as increasing performances in applications unseen during training (Bommasani et al., Reference Bommasani, Hudson, Adeli, Altman, Arora, von Arx, Bernstein, Bohg, Bosselut, Brunskill, Brynjolfsson, Buch, Card, Castellon, Chatterji, Chen, Creel, Davis, Demszky, Donahue, Doumbouya, Durmus, Ermon, Etchemendy, Ethayarajh, Fei-Fei, Finn, Gale, Gillespie, Goel, Goodman, Grossman, Guha, Hashimoto, Henderson, Hewitt, Ho, Hong, Hsu, Huang, Icard, Jain, Jurafsky, Kalluri, Karamcheti, Keeling, Khani, Khattab, Koh, Krass, Krishna, Kuditipudi, Kumar, Ladhak, Lee, Lee, Leskovec, Levent, Li, Li, Ma, Malik, Manning, Mirchandani, Mitchell, Munyikwa, Nair, Narayan, Narayanan, Newman, Nie, Niebles, Nilforoshan, Nyarko, Ogut, Orr, Papadimitriou, Park, Piech, Portelance, Potts, Raghunathan, Reich, Ren, Rong, Roohani, Ruiz, Ryan, R’e, Sadigh, Sagawa, Santhanam, Shih, Srinivasan, Tamkin, Taori, Thomas, Tramèr, Wang, Wang, Wu, Wu, Wu, Xie, Yasunaga, You, Zaharia, Zhang, Zhang, Zhang, Zhang, Zheng, Zhou and Liang2021). Most of the discussed machine learning strategies can be further explored by training foundation models with diverse datasets, thereby enhancing their generalization capabilities. Forest datasets encompass a wide range of scales, ranging from field measurements to estimated world maps (refer to Figure 2), as well as varying resolutions and modalities for different tasks (as outlined in Section 4). The data diversity necessitates the utilization of generalized deep learning architectures. Influenced by the success of large language models (Devlin et al., Reference Devlin, Chang, Lee and Toutanova2019; Radford et al., Reference Radford, Wu, Child, Luan, Amodei and Sutskever2019, Reference Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, Krueger and Sutskever2021; Brown et al., Reference Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever and Amodei2020; Chowdhery et al., Reference Chowdhery, Narang, Devlin, Bosma, Mishra, Roberts, Barham, Chung, Sutton, Gehrmann, Schuh, Shi, Tsvyashchenko, Maynez, Rao, Barnes, Tay, Shazeer, Prabhakaran, Reif, Du, Hutchinson, Pope, Bradbury, Austin, Isard, Gur-Ari, Yin, Duke, Levskaya, Ghemawat, Dev, Michalewski, Garcia, Misra, Robinson, Fedus, Zhou, Ippolito, Luan, Lim, Zoph, Spiridonov, Sepassi, Dohan, Agrawal, Omernick, Dai, Pillai, Pellat, Lewkowycz, Moreira, Child, Polozov, Lee, Zhou, Wang, Saeta, Diaz, Firat, Catasta, Wei, Meier-Hellstern, Eck, Dean, Petrov and Fiedel2022; Hoffmann et al., Reference Hoffmann, Borgeaud, Mensch, Buchatskaya, Cai, Rutherford, Casas, Hendricks, Welbl, Clark, Hennigan, Noland, Millican, Driessche, Damoc, Guy, Osindero, Simonyan, Elsen, Rae, Vinyals and Sifre2022; Driess et al., Reference Driess, Xia, Sajjadi, Lynch, Chowdhery, Ichter, Wahid, Tompson, Vuong, Yu, Huang, Chebotar, Sermanet, Duckworth, Levine, Vanhoucke, Hausman, Toussaint, Greff, Zeng, Mordatch and Florence2023; Touvron et al., Reference Touvron, Lavril, Izacard, Martinet, Lachaux, Lacroix, Rozière, Goyal, Hambro, Azhar, Rodriguez, Joulin, Grave and Lample2023), recent advancements in computer vision have led to the development of models that incorporate multiple modalities and can perform multiple tasks simultaneously. In recent studies, researchers have explored the concept of multitask vision by utilizing natural images (Cheng et al., Reference Cheng, Schwing and Kirillov2021, Reference Cheng, Misra, Schwing, Kirillov and Girdhar2022; Kirillov et al., Reference Kirillov, Mintun, Ravi, Mao, Rolland, Gustafson, Xiao, Whitehead, Berg, Lo, Dollár and Girshick2023; Li et al., Reference Li, Zhang, Xu, Liu, Zhang, Ni and Shum2023a) or by employing text to enhance performance in vision-based tasks (Dancette and Cord, Reference Dancette and Cord2022; Xu et al., Reference Xu, De Mello, Liu, Byeon, Breuel, Kautz and Wang2022; Jain et al., Reference Jain, Li, Chiu, Hassani, Orlov and Shi2023a). In the realm of integrating image and text for performing tasks on both modalities, alternative approaches have been developed to improve performances by benefiting from their combination (Zhu et al., Reference Zhu, Zhu, Li, Wu, Wang, Li, Wang and Dai2022; Li et al., Reference Li, Zhu, Jiang, Zhu, Li, Yuan, Wang, Qiao, Wang, Wang and Dai2023b). Additionally, generalist models have been constructed to be agnostic to specific modalities and tasks (Jaegle et al., Reference Jaegle, Gimeno, Brock, Zisserman, Vinyals and Carreira2021, Reference Jaegle, Borgeaud, Alayrac, Doersch, Ionescu, Ding, Koppula, Zoran, Brock, Shelhamer, Hénaff, Botvinick, Zisserman, Vinyals and Carreira2022), enabling them to handle diverse modalities and tasks with a unified approach. In the field of computer vision, for example, the segment anything model (Kirillov et al., Reference Kirillov, Mintun, Ravi, Mao, Rolland, Gustafson, Xiao, Whitehead, Berg, Lo, Dollár and Girshick2023) has demonstrated the capability to perform instance segmentation by leveraging prompts in conjunction with input images. These architecture frameworks hold a significant value for forest monitoring tasks, enabling the detection, segmentation, and estimation of tree properties over large geographical areas, such as their canopy surface or their aboveground biomass (Tolan et al., Reference Tolan, Yang, Nosarzewski, Couairon, Vo, Brandt, Spore, Majumdar, Haziza, Vamaraju, Moutakani, Bojanowski, Johns, White, Tiecke and Couprie2023; Tucker et al., Reference Tucker, Brandt, Hiernaux, Kariryaa, Rasmussen, Small, Igel, Reiner, Melocik, Meyer, Sinno, Romero, Glennie, Fitts, Morin, Pinzon, McClain, Morin, Porter, Loeffler, Kergoat, Issoufou, Savadogo, Wigneron, Poulter, Ciais, Kaufmann, Myneni, Saatchi and Fensholt2023).

The utilization of foundation models with remote sensing data is still in its infancy. However, promising advances have been made in developing multimodal architectures (Zhang et al., Reference Zhang, Ming, Feng, Liu, He and Zhao2023) and temporal-based approaches (Garnot et al., Reference Garnot, Landrieu and Chehata2021; Garnot and Landrieu, Reference Garnot and Landrieu2021; Tarasiou et al., Reference Tarasiou, Chavez and Zafeiriou2023) specifically tailored for precise tasks in remote sensing applications. Based on the masked autoencoder pretraining method (He et al., Reference He, Chen, Xie, Li, Dollár and Girshick2022), multimodal and multitask architectures have been developed for Earth observation applications, particularly for land use and land cover (LULC) estimation (Cong et al., Reference Cong, Khanna, Meng, Liu, Rozi, He, Burke, Lobell and Ermon2022; Reed et al., Reference Reed, Gupta, Li, Brockman, Funk, Clipp, Funk, Candido, Uyttendaele and Darrell2022; Sun et al., Reference Sun, Wang, Lu, Zhu, Lu, He, Li, Rong, Yang, Chang, He, Yang, Wang, Lu and Fu2022; Tseng et al., Reference Tseng, Zvonkov, Purohit, Rolnick and Kerner2023). These architectures address the challenges posed by data collected from sensors that record diverse physical measurements, such as multispectral or SAR data in remote sensing (Reed et al., Reference Reed, Gupta, Li, Brockman, Funk, Clipp, Funk, Candido, Uyttendaele and Darrell2022; Pan et al., Reference Pan, Gao, Dong and Du2023; Yamazaki et al., Reference Yamazaki, Hanyu, Tran, Garcia, Tran, McCann, Liao, Rainwater, Adkins, Molthan, Cothren and Le2023), as well as in natural images (Themyr et al., Reference Themyr, Rambour, Thome, Collins and Hostettler2023). While different spectral, spatial, and temporal resolutions have been considered in previous works, there remains a lack of exploration regarding the resolution gap between datasets captured by aerial and satellite sensors. The integration of multimodal, multitask, and multiscale architectures is expected to significantly enhance the generalization capabilities of models for forest monitoring tasks at global scale (see Section 2.2.4). By training these algorithms with various types of datasets and specializing them for forest monitoring tasks, they could effectively deliver improved performance across different geographical regions such as for species cover estimation or aboveground biomass estimation (see Section 2.1.3).

3.2.1. Self-supervised learning

Situated in-between supervised and unsupervised learning, the self-supervised learning paradigm involves training a model to reconstruct certain known relationships between or within the datapoints themselves. The resulting model can then be fine-tuned with actual labeled data or directly applied to solve the downstream task. Self-supervised approaches in computer vision include discriminative approaches that distinguish between positive and negative samples while separating their representation (e.g., contrastive learning) (Gidaris et al., Reference Gidaris, Singh and Komodakis2018; Chen et al., Reference Chen, Kornblith, Norouzi and Hinton2020; He et al., Reference He, Fan, Wu, Xie and Girshick2020; Caron et al., Reference Caron, Touvron, Misra, Jégou, Mairal, Bojanowski and Joulin2021; Oquab et al., Reference Oquab, Darcet, Moutakanni, Vo, Szafraniec, Khalidov, Fernandez, Haziza, Massa, El-Nouby, Assran, Ballas, Galuba, Howes, Huang, Li, Misra, Rabbat, Sharma, Synnaeve, Xu, Jegou, Mairal, Labatut, Joulin and Bojanowski2023) and generative approaches that learn representations through reconstruction (Lehtinen et al., Reference Lehtinen, Munkberg, Hasselgren, Laine, Karras, Aittala and Aila2018; He et al., Reference He, Chen, Xie, Li, Dollár and Girshick2022). The utilization of self-supervised learning in remote sensing has experienced significant growth due to the abundance of unlabeled open-access data (Tao et al., Reference Tao, Qi, Guo, Zhu and Li2023). For instance, geolocation of satellite images has been exploited with a contrastive approach (Ayush et al., Reference Ayush, Uzkent, Meng, Tanmay, Burke, Lobell and Ermon2021; Mai et al., Reference Mai, Lao, He, Song and Ermon2023b). Multispectral and SAR data have been reconstructed based on the temporal information (Cong et al., Reference Cong, Khanna, Meng, Liu, Rozi, He, Burke, Lobell and Ermon2022; Yadav et al., Reference Yadav, Nascetti, Azizpour and Ban2022), for multiscale reconstruction (Reed et al., Reference Reed, Gupta, Li, Brockman, Funk, Clipp, Funk, Candido, Uyttendaele and Darrell2022) and for denoising (Dalsasso et al., Reference Dalsasso, Denis and Tupin2021, Reference Dalsasso, Denis and Tupin2022; Meraoumia et al., Reference Meraoumia, Dalsasso, Denis, Abergel and Tupin2023). Emerging cross-modal approaches, encompassing both discriminative (Jain et al., Reference Jain, Wilson and Gulshan2022) and generative (Jain et al., Reference Jain, Schoen-Phelan and Ross2023b) techniques, are being developed to harness the complementary nature of aligned samples. Self-supervised learning will greatly unleash the potential of remote sensing data in the area of forests (Ge et al., Reference Ge, Gu, Su, Lönnqvist and Antropov2023) by learning textural and geometrical structures of forests and trees without labels (see Sections 2.2.1 and 2.2.4).

3.2.2. Weakly supervised learning

Obtaining precise and detailed annotations, for example, for tree crown instance segmentation, can be both costly and time-consuming. Although self-supervised learning aims to learn representations from pretext tasks, it still necessitates precise annotations for fine-tuning the model in a downstream task. In cases where precise annotations are not available, coarse-grained and potentially inaccurate annotations, or even single point locations, can be utilized as weak labels in weakly supervised learning approaches (Zhou, Reference Zhou2018). Given their cost-effectiveness and efficiency, computer vision methods have been developed to leverage weak annotations while addressing their inherent inaccuracies (Zhou, Reference Zhou2018). Weakly supervised learning has been explored in the realm of object location (Oquab et al., Reference Oquab, Bottou, Laptev and Sivic2015), object relationship estimation (Peyre et al., Reference Peyre, Sivic, Laptev and Schmid2017), instance segmentation (Ahn et al., Reference Ahn, Cho and Kwak2019), and contrastive learning (Zheng et al., Reference Zheng, Wang, You, Qian, Zhang, Wang and Xu2021). Obtaining high-quality annotations for remote sensing data is particularly difficult due to their poor spatial resolution or the physics of the sensors used. Weakly supervised learning has therefore been investigated for Earth observation tasks including object detection (Han et al., Reference Han, Zhang, Cheng, Guo and Ren2015; Zhang et al., Reference Zhang, Han, Cheng, Liu, Bu and Guo2015; Yao et al., Reference Yao, Feng, Han, Cheng and Guo2021), LULC semantic segmentation (Yao et al., Reference Yao, Han, Cheng, Qian and Guo2016; Wang et al., Reference Wang, Chen, Xie, Azzari and Lobell2020b), and plant traits regression (Schiller et al., Reference Schiller, Schmidtlein, Boonman, Moreno-Martínez and Kattenborn2021; Cherif et al., Reference Cherif, Feilhauer, Berger, Dao, Ewald, Hank, He, Kovach, Lu, Townsend and Kattenborn2023). Recently, weakly supervised methods have been investigated in the areas of tree classification (Illarionova et al., Reference Illarionova, Trekin, Ignatiev and Oseledets2021), tree counting (Amirkolaee et al., Reference Amirkolaee, Shi and Mulligan2023), tree detection (Aygunes et al., Reference Aygunes, Cinbis and Aksoy2021), and tree segmentation (Gazzea et al., Reference Gazzea, Kristensen, Pirotti, Ozguven and Arghandeh2022) using multispectral data (see Section 2.2.4).

3.2.3. Active learning

Even highly precise and fine-grained annotations are generally less useful if present in only small quantities. To address this limitation, active learning strategies have been developed to identify and select the optimal way to select a small set of training datapoints to label (Cohn et al., Reference Cohn, Ghahramani and Jordan1996). These strategies for sample selection often rely on estimating the uncertainty of a model (Gal et al., Reference Gal, Islam and Ghahramani2017), for instance, using variational approaches (Sinha et al., Reference Sinha, Ebrahimi and Darrell2019) or estimated with a loss function (Yoo and Kweon, Reference Yoo and Kweon2019). They have demonstrated their effectiveness in scenarios where the amount of labeled data is limited, particularly in the context of image classification (Gal et al., Reference Gal, Islam and Ghahramani2017; Sinha et al., Reference Sinha, Ebrahimi and Darrell2019; Yoo and Kweon, Reference Yoo and Kweon2019), object detection (Roy et al., Reference Roy, Unmesh and Namboodiri2019), and semantic segmentation (Siddiqui et al., Reference Siddiqui, Valentin and Niessner2020). Active learning has also been investigated for remote sensing applications, including classification (Tuia et al., Reference Tuia, Volpi, Copa, Kanevski and Munoz-Mari2011), object detection (Qu et al., Reference Qu, Du, Cao, Guan and Zhao2020), and LULC semantic segmentation with hyperspectral data (Li et al., Reference Li, Bioucas-Dias and Plaza2010; Li et al., Reference Li, Bioucas-Dias and Plaza2011; Zhang et al., Reference Zhang, Pasolli, Crawford and Tilton2016). Its application would be helpful for forest monitoring to optimize and create relevant human annotations (see Section 2.2.4).

3.2.4. Few-shot learning

Yet another approach to limited data availability is few-shot learning, which refers to efficient fine-tuning of a pretrained model using only a few annotated datapoints. Few-shot learning has been approached from different perspectives, considering the comparison between the small annotated dataset and the data used for pretraining the model—for example, by quantifying the similarities between these datasets (Vinyals et al., Reference Vinyals, Blundell, Lillicrap, Kavukcuoglu and Wierstra2016), constructing mixtures of feature embeddings (Snell et al., Reference Snell, Swersky and Zemel2017), or adapting the optimization scheme through meta-learning (Finn et al., Reference Finn, Abbeel and Levine2017). Motivated by the limited availability of annotations, applications of few-shot learning in remote sensing tasks have been investigated. For instance, methods based on feature similarity (Alajaji et al., Reference Alajaji, Alhichri, Ammour and Alajlan2020; Zhang et al., Reference Zhang, Bai, Wang, Bai and Li2020; Alosaimi et al., Reference Alosaimi, Alhichri, Bazi, Ben Youssef and Alajlan2023) and metric learning (Liu et al., Reference Liu, Yu, Yu, Zhang, Wan and Wang2019), aiming at separating representations in an embedding space, have been explored for LULC classification with either multispectral or hyperspectral data. Objects have also been detected by learning meta features (Deng et al., Reference Deng, Li and Fang2022). Metric learning techniques have also been utilized in the context of few-shot learning for semantic segmentation tasks (Jiang et al., Reference Jiang, Zhou and Li2022) or meta learning with multispectral and SAR data (Rußwurm et al., Reference Rußwurm, Wang, Körner and Lobell2020). Few-shot learning has been explored for tree species classification using feature similarity (Chen et al., Reference Chen, Tian, Chai, Zhang and Chen2021) and would be beneficial to recognize a species or estimate the characteristics of a tree with minimal manual annotations (see Sections 2.2.2 and 2.2.4).

3.2.5. Zero-shot learning

The machine learning community has also shown interest in developing methods for training algorithms to differentiate unseen classes without any explicitly annotated samples at all, which is known as zero-shot learning (Xian et al., Reference Xian, Lampert, Schiele and Akata2018). In order to categorize unseen classes, the task of zero-shot learning has been accomplished by projecting image and word embeddings (Socher et al., Reference Socher, Ganjoo, Sridhar, Bastani, Manning and Ng2013) or known semantic attributes (Lampert et al., Reference Lampert, Nickisch and Harmeling2014) into a shared space. Zero-shot learning has also been investigated by incorporating a mixture of embeddings from the source domain before computing similarities with the target domain, which includes the unseen classes (Zhang and Saligrama, Reference Zhang and Saligrama2015). Generative approaches have been developed to create visual feature embeddings of unseen classes from word embeddings for zero-shot semantic segmentation (Bucher et al., Reference Bucher, Vu, Cord and Pérez2019). Zero-shot learning has also garnered attention in remote sensing applications, including combining multispectral data and word embeddings for classification tasks (Li et al., Reference Li, Lu, Wang, Xiang and Wen2017, Reference Li, Kong, Zhang, Tan and Chen2021) and initial exploration of applying zero-shot learning to classify hyperspectral data (Freitas et al., Reference Freitas, Silva and Silva2022). Generative approaches have also been used with remote sensing data to create visual embeddings from word embeddings (Li et al., Reference Li, Zhang, Wang, Lin and Zhang2022b). Zero-shot learning presents a promising approach for forest monitoring, enabling the adaptation of models in regions where previously unseen species are encountered (see Sections 2.2.2 and 2.2.4). By leveraging tree taxonomy hierarchy and meta characteristics to align with visual embeddings (Sumbul et al., Reference Sumbul, Cinbis and Aksoy2018), a vast research potential emerges. Notably, utilizing foundation models that have demonstrated strong zero-shot learning capabilities (Brown et al., Reference Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever and Amodei2020; Radford et al., Reference Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, Krueger and Sutskever2021) further enhances this potential. In the following paragraph, we will explore methods concerning domain-specific objectives, focusing on the consideration of physical and biological constraints and their applications.

3.3.1. Constraints on data

Depending on the domain of application, the outputs of a machine learning pipeline may have specific constraints that must be satisfied if the answer is to be useful or even possible. For example, climate variables may need to obey physical laws such as conservation of energy, engineered systems may need to obey the laws of mechanics, and so forth. Machine learning models to work with such variables have increasingly been designed with soft constraints (Ouaknine et al., Reference Ouaknine, Newson, Pérez, Tupin and Rebut2021a; Harder et al., Reference Harder, Watson-Parris, Stier, Strassel, Gauger and Keuper2022), which impose penalties for constraint violation, or hard constraints (Donti et al., Reference Donti, Rolnick and Kolter2021; Geiss and Hardin, Reference Geiss and Hardin2021; Harder et al., Reference Harder, Ramesh, Hernandez-Garcia, Yang, Sattigeri, Szwarcman, Watson and Rolnick2023), where the constraints are strictly enforced by the design of the algorithm. Compared to physics- and engineered-based constraints, fewer authors have to date integrated biological constraints into ML algorithms. Dynamics of biological systems have been included in a deep learning optimization scheme as hard constraints from ordinary differential equations (Yazdani et al., Reference Yazdani, Lu, Raissi and Karniadakis2020). There are potential opportunities for incorporating biological constraints in forest monitoring by considering phenological (Richardson et al., Reference Richardson, Hufkens, Milliman and Frolking2018) or biophysical traits, or ecosystem properties (see Section 2.1.5), for example, by considering the ratio of tree height and canopy size. These constraints could be particularly valuable in tasks such as semantic segmentation or biomass estimation.

Domain-specific constraints on data may also pose opportunities for improving the design of machine learning models. The design of deep learning model architectures can incorporate considerations for, or reconstruction of, physical properties. For instance, a physics-informed architecture has been developed for super-resolution in turbulent flows, incorporating partial differential equations as a form of regularization (Jiang et al., Reference Jiang, Esmaeilzadeh, Azizzadenesheli, Kashinath, Mustafa, Tchelepi, Marcus, Prabhat and Anandkumar2020). Similarly, RADAR-based architectures have been created to reconstruct physical properties for scene understanding in the context of autonomous driving (Ouaknine et al., Reference Ouaknine, Newson, Pérez, Tupin and Rebut2021a; Rebut et al., Reference Rebut, Ouaknine, Malik and Pérez2022). Leveraging the properties of multiple sensors has also been employed to fuse their representations (Ouaknine, Reference Ouaknine2022) or to generate annotations from one modality to another (Ouaknine et al., Reference Ouaknine, Newson, Rebut, Tupin and Perez2021b; Schiefer et al., Reference Schiefer, Schmidtlein, Frick, Frey, Klinke, Zielewska-Büttner, Junttila, Uhl and Kattenborn2023). In remote sensing, self-supervised learning has benefited from SAR physical properties by considering a pretext denoising task (Dalsasso et al., Reference Dalsasso, Denis and Tupin2021; Meraoumia et al., Reference Meraoumia, Dalsasso, Denis, Abergel and Tupin2023), or by separating and reconstructing the real from the imaginary part of the signal (Dalsasso et al., Reference Dalsasso, Denis and Tupin2022). Such methods could also be explored by exploiting various sensors to learn representations of forests and trees (see Section 2.2.4).

3.3.2. Uncertainty quantification

Biological phenomena adhere to intricate rules that are challenging to estimate and often exhibit inherent uncertainties. The estimation of prediction uncertainty aids in obtaining a better understanding of the strengths and limitations of a machine learning model. The overall uncertainty of these models comprises both aleatoric and epistemic uncertainties (Gal, Reference Gal2016). They both can be distinguished based on their origins. Aleatoric uncertainty arises from the inherent noise present in the data and label distributions, while epistemic uncertainty is associated with the model itself, encompassing its estimated parameters and structural characteristics. Approaches have been devised to estimate the uncertainties of deep neural networks, for example, by using a Bayesian approach such as Monte Carlo dropout (Gal and Ghahramani, Reference Gal and Ghahramani2016), by using adversarial training combined with model ensembles (Lakshminarayanan et al., Reference Lakshminarayanan, Pritzel and Blundell2017), by predicting the uncertainty distribution (Malinin and Gales, Reference Malinin and Gales2018), or by learning an auxiliary confidence score from the data (Corbière et al., Reference Corbière, Thome, Bar-Hen, Cord and Pérez2019; Corbière, Reference Corbière2022). Similar methods have been applied to estimate uncertainty in remote sensing data for crop yield estimation (Ma et al., Reference Ma, Zhang, Kang and özdoğan2021b) or road segmentation (Haas and Rabus, Reference Haas and Rabus2021). The quantification of uncertainties in forest monitoring methods has been carried out to assess both aleatoric and epistemic uncertainties (see Section 2.2). This is commonly performed to evaluate the uncertainty of predictions on large-scale maps, utilizing low-resolution satellite data. The uncertainty of plant functional type has been studied for classification in Siberia (Ottlé et al., Reference Ottlé, Lescure, Maignan, Poulter, Wang and Delbart2013). Estimating the uncertainty of aboveground biomass has also been conducted to establish a range of estimated values in carbon stock maps (Patterson et al., Reference Patterson, Healey, Ståhl, Saarela, Holm, Andersen, Dubayah, Duncanson, Hancock, Armston, Kellner, Cohen and Yang2019; Santoro et al., Reference Santoro, Cartus, Carvalhais, Rozendaal, Avitabile, Araza, de Bruin, Herold, Quegan, Rodriguez-Veiga, Balzter, Carreiras, Schepaschenko, Korets, Shimada, Itoh, Martínez, Cavlovic, Cazzolla Gatti, da Conceiçao Bispo, Dewnath, Labrière, Liang, Lindsell, Mitchard, Morel, Pacheco Pascagaza, Ryan, Slik, Vaglio Laurin, Verbeeck, Wijaya and Willcock2021) (see Section 2.1.3). To quantify uncertainty, these methods utilize standard deviations or output probabilities of the model. Recent studies have taken a step further in estimating tree carbon stocks in semi-arid sub-Saharan Africa north of the Equator by combining uncertainty from both allometric equations and predicted crown segmentation, utilizing field measurements (Tucker et al., Reference Tucker, Brandt, Hiernaux, Kariryaa, Rasmussen, Small, Igel, Reiner, Melocik, Meyer, Sinno, Romero, Glennie, Fitts, Morin, Pinzon, McClain, Morin, Porter, Loeffler, Kergoat, Issoufou, Savadogo, Wigneron, Poulter, Ciais, Kaufmann, Myneni, Saatchi and Fensholt2023). There has been limited application of advanced uncertainty quantification methods, whether associated with the data or the predictive model, in the context of forest monitoring.

Despite the extensive application of the presented machine learning techniques in remote sensing, their utilization for forest monitoring has been relatively limited. This presents numerous opportunities to gain deeper insights into the composition of forests while achieving generalization at a large scale. However, it is crucial to have access to high-quality, diverse, and sufficient datasets in order to effectively explore machine learning strategies. In the following section, we will review open-access forest datasets, providing information on their size, tasks, scale, and modalities.