Impact Statement

By downscaling coarse weather variables, we can produce high-resolution datasets that capture the finer dynamics of the weather. As training with high-resolution datasets is compute and time-intensive, zero-shot downscaling can accelerate atmospheric science workflows as it avoids the need to collect training data at the highest resolution. This work investigates neural operator-based zero-shot weather downscaling methods, as neural operators have shown promising zero-shot super-resolution results on dynamical systems emulation. Our results highlight the importance of high-quality spatial domain features over resolution-invariant spectral features for zero-shot weather downscaling; however, all models show potential for improvement.

2. Related work

Weather downscaling with deep learning Deep learning models have recently shown promise at weather downscaling tasks such as precipitation downscaling (Chaudhuri and Robertson, Reference Chaudhuri and Robertson2020; Watson et al., Reference Watson, Wang, Lynar and Weldemariam2020; Harris et al., Reference Harris, McRae, Chantry, Dueben and Palmer2022). These end-to-end differentiable approaches directly learn to map low-resolution inputs to high-resolution outputs. The most popular approaches such as the super-resolution Convolutional Neural Network (SRCNN) (Dong et al., Reference Dong, Loy, He and Tang2015) are based on architectures introduced by the computer vision community for super-resolution (Wang et al., Reference Wang, Chen and Hoi2020). These models are used as key baselines in our experiments, thus, we describe them in more detail in Section 4. (Groenke et al., Reference Groenke, Madaus and Monteleoni2020) have used normalizing flow models (Rezende and Mohamed, Reference Rezende and Mohamed2015) to perform climate downscaling in an unsupervised manner. (Harder et al., Reference Harder, Hernandez-Garcia, Ramesh, Yang, Sattegeri, Szwarcman, Watson and Rolnick2023) add physics-based constraints in the learning process of the convolutional neural network (CNN) and generative adversarial network (GAN) models for downscaling variables such as water content and temperature. Our work does not include these constraints as our focus is solely on studying the neural operator layers for the downscaling task. Works (Stengel et al., Reference Stengel, Glaws, Hettinger and King2020; Buster et al., Reference Buster, Benton, Glaws and King2024) downscale renewable energy datasets, such as wind and solar, as energy system planning depends on high-resolution estimates of these resources. Buster et al. (Reference Buster, Benton, Glaws and King2024) use a custom GAN (Stengel et al., Reference Stengel, Glaws, Hettinger and King2020) trained on Global Climate Models (GCMs) projections to generate high-resolution spatial and temporal features capturing small-scale details otherwise lost in the coarse GCM models. (Benton et al., Reference Benton, Buster, Pinchuk, Glaws, King, Maclaurin and Chernyakhovskiy2024) also use the custom GAN (Stengel et al., Reference Stengel, Glaws, Hettinger and King2020) for spatiotemporal downscaling of wind data to learn a mapping from low-resolution European Centre for Medium-Range Weather Forecasts Reanalysis version 5 (ERA5) (Hersbach et al., Reference Hersbach, Bell, Berrisford, Hirahara, Horányi, Muñoz-Sabater, Nicolas, Peubey, Radu, Schepers, Simmons, Soci, Abdalla, Abellan, Balsamo, Bechtold, Biavati, Bidlot, Bonavita, Chiara, Dahlgren, Dee, Diamantakis, Dragani, Flemming, Forbes, Geer, Haimberger, Healy, Hogan, Hólm, Janisková, Keeley, Laloyaux, Lopez, Lupu, Radnoti, Rosnay, Rozum, Vamborg, Villaume and Thépaut2020) to high-resolution Wind Integration National Dataset Toolkit (WTK) (Draxl et al., Reference Draxl, Clifton, Hodge and McCaa2015). (Tran et al., Reference Tran, Robinson, Rasheed, San, Tabib and Kvamsdal2020) use ESRGAN (Wang et al., Reference Wang, Yu, Wu, Gu, Liu, Dong, Qiao and Loy2018) for super-resolution of the wind field. Through this work, we also perform downscaling of wind data using ERA5 and WTK datasets, but with the goal of investigating the utility of neural operator models for the weather downscaling task with a specific emphasis on zero-shot downscaling.

Related benchmarks for weather downscaling SuperBench (Pu Ren et al., Reference Pu Ren, Subramanian, San, Lukic and Mahoney2023) introduces super-resolution datasets and benchmarks for scientific applications, such as fluid flow, cosmology, and weather downscaling. Their work compares various deep learning methods and analyzes the physics-preserving properties of these models. RainNet (Chen et al., Reference Chen, Feng, Liu, Ni, Lu, Tong and Liu2022) is one of the first large-scale datasets for the task of spatial precipitation downscaling, spanning over 17 years and covering important meteorological phenomena. Their work also presents an extensive evaluation of many deep learning models. WiSoSuper (Kurinchi-Vendhan et al., Reference Kurinchi-Vendhan, Lütjens, Gupta, Werner and Newman2021) is a benchmark for wind and solar super-resolution. The dataset released by WiSoSuper is based on the National Renewable Energy Laboratory’s (NREL’s) WTK and National Solar Radiation Database (NSRDB) (Sengupta et al., Reference Sengupta, Xie, Lopez, Habte, Maclaurin and Shelby2018) datasets. They compare generative models introduced in (Stengel et al., Reference Stengel, Glaws, Hettinger and King2020) with other GAN and CNN-based models. In contrast to these benchmarking efforts, our work benchmarks models for weather downscaling but with a focus on neural operator models and zero-shot weather downscaling. In a concurrent work focusing on climate downscaling (Prasad et al., Reference Prasad, Harder, Yang, Sattegeri, Szwarcman, Watson and Rolnick2024), CNNs, transformers (Alexey, Reference Alexey2020), and neural-operator-based models are compared in terms of their ability to pretrain on diverse climate datasets so as to learn transferrable representations across multiple climate variables and spatial regions.

Arbitrary-scale super-resolution Arbitrary-scale super-resolution (ASSR) has been gaining in popularity (Liu et al., Reference Liu, Li, Shang, Liu, Wan, Feng and Timofte2023) in the study of super-resolution in computer vision. ASSR involves training a single deep learning model to super-resolve images to arbitrarily high (potentially non-integral) upsampling factors. Unlike ASSR, which broadly considers any image datasets relevant to computer vision, our work focuses specifically on zero-shot weather downscaling problems which possess unique challenges due to, for example, multi-scale physical phenomena. Super-resolution neural operator (SRNO) (Wei and Zhang, Reference Wei and Zhang2023) and the recent Hierarchical Neural Operator Transformer (HiNOTE) model (Luo et al., Reference Luo, Qian and Yoon2024) are two advanced deep learning models built with neural operator layers that show promising ASSR performance. SRNO proposes treating the LR and HR images as approximations of finite dimensional continuous functions with different grid sizes and learns a mapping between them. They introduce an advanced LR image encoding framework and kernel integral layers to learn an expressive mapping. HiNOTE is a hierarchical hybrid neural operator-transformer model. We did not include SRNO and HiNOTE in our experiments as we focus on a simple framework (Figure 1) where downscaling performance can be easily attributed to the neural operator components under study.

Figure 1.

Overview of the neural operator and non-neural-operator zero-shot weather downscaling approaches. We show 5x to 15x zero-shot downscaling as an example. (a,b) For neural operators, the interpolation scale factor is the same as the upsampling factor, e.g., the bicubic interpolation layer upsamples to 5x during training and 15x during evaluation. (c) For regular neural networks (e.g., SwinIR), the model is trained to output at 5x (e.g., using a learnable upsampler such as sub-pixel convolution). At test time, the model generates a 5x output which is then interpolated 3x more to produce the final 15x HR output.

3. Background

Neural operators Operator learning models, such as neural operators (Kovachki et al., Reference Kovachki, Li, Liu, Azizzadenesheli, Bhattacharya, Stuart and Anandkumar2023), are composed of layers that learn mappings between infinite-dimensional function spaces. In doing so, they approximately learn a continuous operator, which can be realized at any arbitrary grid discretization of the input and output. Thus, neural operators do not depend on the discretization of the grid they are trained on, and we expect them to generalize to grid resolutions different than the ones they are trained on. (Li et al., Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021) introduced Fourier neural operators (FNOs), expressing neural operators as a combination of linear integral operators to incorporate the non-local properties of the solution operator with Fourier Transform and non-linear local activation functions (Kovachki et al., Reference Kovachki, Li, Liu, Azizzadenesheli, Bhattacharya, Stuart and Anandkumar2023), which helps to model non-linear systems and their high-frequency modes. They show improved performance over convolution-based models for complex non-linear PDEs such as the Navier–Stokes equation. With the Fourier layers, the parameters are learned in the Fourier domain, which enables FNOs to be invariant to the grid discretization or resolution. Because neural operators such as FNOs learn resolution-invariant approximations of continuous operators, we aim to understand whether this provides advantages for zero-shot weather downscaling.

(Yang et al., Reference Yang, Hernandez-Garcia, Harder, Ramesh, Sattegeri, Szwarcman, Watson and Rolnick2023) adapt FNOs (Li et al., Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021) to perform downscaling on ERA5 and PDE data. Their proposed model, which they refer to as DFNO, outperforms CNN and GAN-based models. They also evaluate zero-shot downscaling on unseen upsampling factors to observe the model’s zero-shot generalization potential. We adapt and expand this downscaling pipeline in our benchmarking study. Our work differs from this paper as we investigate higher upsampling factors (8x and 15x) for training and zero-shot evaluations as opposed to 2x and 4x in their work. We also create a realistic set of experiments that includes LR and HR data sourced from different simulations (ERA5 to WTK downscaling, as described in Section 4.2) and compare various neural operator approaches against strong baselines including powerful transformers.

Weather downscaling In weather downscaling, we are given a snapshot of LR weather data (e.g., an image) with a goal of upsampling this data to a higher target resolution. Mathematically, in the standard downscaling problem, we have the LR input grid $ \mathbf{x}\in {\mathrm{\mathbb{R}}}^{h\times w\times c} $ , and a target, HR output, $ \mathbf{y}\in {\mathrm{\mathbb{R}}}^{h^{\prime}\times {w}^{\prime}\times c} $ , where $ h,w\in \unicode{x2115} $ , $ c $ is the number of atmospheric variables, and $ h\times w $ is a lower resolution than $ {h}^{\prime}\times {w}^{\prime } $ . Deep-learning-based downscaling techniques introduced in Section 2 learn an approximation $ f $ between two finite-dimensional vector spaces $ f:{\mathrm{\mathbb{R}}}^{h\times w\times c}\to {\mathrm{\mathbb{R}}}^{h^{\prime}\times {w}^{\prime}\times c} $ . We refer to the setting where models are trained and tested on the same upsampling factor as standard weather downscaling. In this work, we restrict our focus to only static downscaling problems, i.e., each snapshot represents a single instant in time.

Zero-shot weather downscaling In our work, we wish to evaluate the extent to which downscaling models built with resolution-invariant neural operator layers generalize when tested on previously unseen, higher upsampling factors compared to approaches without such layers. The simplest way to obtain an HR image at any arbitrarily fine discretization is a non-learned interpolation scheme such as bicubic interpolation.

We are looking into neural-operator-based downscaling models that learn a mapping $ {\mathcal{G}}^{\dagger }:{\mathrm{\mathbb{R}}}^{h\times w\times c}\to \mathcal{U} $ from $ \mathbf{x}\in {\mathrm{\mathbb{R}}}^{h\times w\times c} $ to a function $ u\in \mathcal{U} $ . We aim to obtain HR outputs $ \mathbf{y}\in {\mathrm{\mathbb{R}}}^{h^{\prime}\times {w}^{\prime}\times c} $ from a discretization of $ u $ , where $ \mathcal{U} $ is an infinite-dimensional function space (Li et al., Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021; Yang et al., Reference Yang, Hernandez-Garcia, Harder, Ramesh, Sattegeri, Szwarcman, Watson and Rolnick2023). A neural-operator-based downscaling framework (Yang et al., Reference Yang, Hernandez-Garcia, Harder, Ramesh, Sattegeri, Szwarcman, Watson and Rolnick2023) (Figure 1b) learns a parametric approximation of a mapping from the finite-dimensional LR input space to the infinite-dimensional space, $ {G}_{\theta}\left(\mathbf{x}\right):{\mathrm{\mathbb{R}}}^{h\times w\times c}\to \mathcal{U} $ , as an approximation of $ {\mathcal{G}}^{\dagger } $ such that $ {G}_{\theta}\left(\mathbf{x}\right):= {\mathcal{F}}_{\theta}\left({T}^{-1}\left(\hskip0.2em ,{f}_{\theta}\left(\mathbf{x}\right)\right)\right) $ , with $ \theta $ as the parameters of the model. It is comprised of (a) neural network layers that first learn to map LR inputs to an embedding vector, $ {f}_{\theta }:{\mathrm{\mathbb{R}}}^{h\times w\times c}\to {\mathrm{\mathbb{R}}}^d $ , (b) a discretization inversion operator that converts the vector to a function ( $ e\in \mathrm{\mathcal{E}} $ ) with $ {T}^{-1}:{\mathrm{\mathbb{R}}}^d\to \mathrm{\mathcal{E}}\left(D;{\mathrm{\mathbb{R}}}^{d_e}\right) $ , and (c) neural operator layers $ {\mathcal{F}}_{\theta }:\mathrm{\mathcal{E}}\to \mathcal{U} $ that learn to map the function to another function, which can be discretized to produce the HR output $ \mathbf{y}\in {\mathrm{\mathbb{R}}}^{h^{\prime}\times {w}^{\prime}\times c} $ . We refer to these approaches as downscaling NO models (e.g. DFNO (Yang et al., Reference Yang, Hernandez-Garcia, Harder, Ramesh, Sattegeri, Szwarcman, Watson and Rolnick2023)). In order to use vanilla FNOs without resolution-dependent neural network layers (a) (as seen in Figure 1(a)), we learn $ {G}_{\theta}\left(\mathbf{x}\right):= {\mathcal{F}}_{\theta}\left({T}^{-1}\left(\mathbf{x}\right)\right) $ . Several improvements have since been proposed over FNOs (Guibas et al., Reference Guibas, Mardani, Li, Tao, Anandkumar and Catanzaro2021; Rahman et al., Reference Rahman, Ross and Azizzadenesheli2023; Raonic et al., Reference Raonic, Molinaro, De Ryck, Rohner, Bartolucci, Alaifari, Mishra, de Bézenac, Oh, Naumann, Globerson, Saenko, Hardt and Levine2023) which we include in our downscaling study and describe in further detail later (Section 4).

4. Methodology

We use two experimental setups to compare the performance of neural operators and non-neural-operator-based methods at both standard and zero-shot weather downscaling. First, we downscale ERA5 data, where we learn a mapping from coarsened LR ERA5 to HR ERA5. In our second experiment, we downscale from LR ERA5 to HR WTK. We expect the second task to be more challenging as it presents a more realistic downscaling scenario where the LR inputs belong to a different simulation than the HR data. Thus, we do not assume the LR is a coarsened version of the HR (Pu Ren et al., Reference Pu Ren, Subramanian, San, Lukic and Mahoney2023; Benton et al., Reference Benton, Buster, Pinchuk, Glaws, King, Maclaurin and Chernyakhovskiy2024). The coarse simulation in this case differs from the fine simulation in terms of modeling assumptions and physics. The performance of our downscaling models for this complex task provides important insights into their capability to infer unseen fine-scale physical phenomena and to be used in an operational context.

Downscaling neural operator models We compare (vanilla) FNO with downscaling FNO (DFNO), downscaling U-shaped neural operator (Rahman et al., Reference Rahman, Ross and Azizzadenesheli2023) (DUNO), downscaling convolutional neural operator (Raonic et al., Reference Raonic, Molinaro, De Ryck, Rohner, Bartolucci, Alaifari, Mishra, de Bézenac, Oh, Naumann, Globerson, Saenko, Hardt and Levine2023) (DCNO), and downscaling adaptive Fourier neural operator (Guibas et al., Reference Guibas, Mardani, Li, Tao, Anandkumar and Catanzaro2021) (DAFNO). The downscaling (D) models are based on (Yang et al., Reference Yang, Hernandez-Garcia, Harder, Ramesh, Sattegeri, Szwarcman, Watson and Rolnick2023) (as described in Section 3) with FNO, UNO, CNO, and AFNO as the neural operator layers in the modeling framework. We show details of this model in Figure 1(b). The low-resolution (LR) image first passes through a set of Residual-in-Residual Dense Block (RRDB) blocks, where an RRDB block is composed of multiple levels of residual and dense networks as introduced in ESRGAN (Wang et al., Reference Wang, Yu, Wu, Gu, Liu, Dong, Qiao and Loy2018). Then, the embedding is interpolated corresponding to the upsampling factor using bicubic interpolation to obtain a high-resolution output. Finally, this goes through neural operator layers to produce the final downscaled HR image. We can think of this last stage as a post-processing step over the features extracted by the RRDB layers followed by the interpolation. The number of RRDB blocks is a hyperparameter tuned during training. All the Downscaling (D) models are trained with the mean-squared-error (MSE) loss. To perform zero-shot downscaling with either the FNO or DXNO (e.g. DFNO) models at the test time on higher upsampling factors than the ones seen during the training, we use the interpolation layer to increase the resolution by the corresponding higher upsampling factor.

The four neural operator models we compare are:

1. 1. FNO (Li et al., Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021) FNO uses a combination of linear integral operators and non-linear local activation functions to learn the operator mapping on complex PDEs such as the Navier–Stokes equation. We follow the original implementation of FNO from (Li et al., Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021) for the FNO model. Figure 1(a) shows the vanilla FNO model as used in our downscaling pipeline where FNO is post-processing the interpolation output.

2. 2. UNO (Rahman et al., Reference Rahman, Ross and Azizzadenesheli2023) is introduced as a deep and memory-efficient architecture that allows for faster training than FNOs. This neural operator has a U-shaped architecture, comprising an encoder-decoder framework built with neural operator layers to learn mappings between function spaces over different domains.

3. 3. AFNO (Guibas et al., Reference Guibas, Mardani, Li, Tao, Anandkumar and Catanzaro2021), a transformer model that uses FNOs for efficient token-mixing instead of the traditional self-attention layers. AFNOs have been used as an integral part of FourCastNet (Pathak et al., Reference Pathak, Subramanian, Harrington, Raja, Chattopadhyay, Mardani, Kurth, Hall, Li, Azizzadenesheli, Hassanzadeh, Kashinath and Anandkumar2022), a data-driven weather forecasting model.

4. 4. CNO (Raonic et al., Reference Raonic, Molinaro, De Ryck, Rohner, Bartolucci, Alaifari, Mishra, de Bézenac, Oh, Naumann, Globerson, Saenko, Hardt and Levine2023) adapts CNNs for learning operators and processing the inputs and outputs as function spaces. They adopt a UNet (Ronneberger et al., Reference Ronneberger, Fischer and Brox2015)-based modeling framework to learn a mapping between bandlimited functions (Vetterli et al., Reference Vetterli, Kovačević and Goyal2014), which supports resolution invariance within the captured frequency band and can help them to learn operators that reduce aliasing errors (Bartolucci et al., Reference Bartolucci, de Bézenac, Raonic, Molinaro, Mishra and Alaifari2024) which occurs when the neural operator models try to learn a continuous operator on a finite, discretized grid.

The hyperparameters and model training details are presented in Appendix A.2. Additionally, we also conducted a detailed analysis of how the frequency cutoff hyperparameter affects the downscaling performance of the FNO, see Appendix A.4.

Baseline models We compare all the neural operator models with five baselines: (1) bicubic interpolation, (2) SRCNN, (3) ESRGAN, (4) EDSR, and (5) SwinIR. Super-resolution convolutional neural network or SRCNN (Dong et al., Reference Dong, Loy, He and Tang2015) is the first CNN-based model to perform single image or spatial super-resolution. SRCNN first upsamples the LR input with bicubic interpolation followed by lightweight CNN layers to obtain the HR image. Enhanced deep super-resolution network (EDSR) (Lim et al., Reference Lim, Son, Kim, Nah and Lee2017) introduces deep residual CNN networks to do super-resolution, where the CNN layers are followed by an upsampling block performing sub-pixel convolution (Shi et al., Reference Shi, Caballero, Huszár, Totz, Aitken, Bishop, Rueckert and Wang2016). ESRGAN (Wang et al., Reference Wang, Yu, Wu, Gu, Liu, Dong, Qiao and Loy2018) is a GAN-based architecture where the generator is composed of many RRDB blocks. The model is trained with pixel and perceptual loss along with adversarial loss, the perceptual loss minimizes the errors in the feature space and helps improve the visual quality of the generated super-resolved image (Ledig et al., Reference Ledig, Theis, Huszár, Caballero, Cunningham, Acosta, Aitken, Tejani, Totz, Wang and Shi2017). Swin Transformer for Image Restoration (SwinIR) (Liang et al., Reference Liang, Cao, Sun, Zhang, Van Gool and Timofte2021) has the advantages of both the CNN and Swin transformer (Liu et al., Reference Liu, Lin, Cao, Hu, Wei, Zhang, Lin and Guo2021) layers. It captures long-range dependencies and learns robust features to improve super-resolution performance with the residual Swin Transformer blocks (RSTB) which is composed of many Swin Transformer layers stacked together with residual connections.

We refer to the implementation of these models as released in the SuperBench (Pu Ren et al., Reference Pu Ren, Subramanian, San, Lukic and Mahoney2023) work, hyperparameter details are added in Appendix A.2. Unlike neural operators, these models have architectures that expect their inputs and outputs to have the same grid resolution at both training and test time. Thus, we add a bicubic interpolation module on the output obtained from these models to produce outputs at higher upsampling factors than seen during training for our zero-shot downscaling experiments (Figure 1(c)).

See Appendix A.3, Table 4 comparing the parameter count and training wall-clock time of all the neural operator and non-neural-operator-based models. Notably, we tested the vanilla FNO downscaling framework (Figure 1(a)), by moving the bicubic interpolation module after the FNO layers (as in baseline models, Figure 1(c)), which led to worse performance compared to our proposed pipeline with interpolation before the FNO layers.

4.2. ERA5 to WTK downscaling

For this second experiment, we focus on downscaling from LR ERA5 to HR WTK. It should be noted that the LR data in this setup is not obtained by coarsening the HR data but comes from another simulation. We include this experiment to observe the performance of neural operators in a challenging and more realistic setup; for e.g., where ERA5 serves as boundary conditions for dynamical downscaling with a Numerical Weather Prediction model (Benton et al., Reference Benton, Buster, Pinchuk, Glaws, King, Maclaurin and Chernyakhovskiy2024). The HR WTK dataset is available over two regions in the US at a 2-km resolution (Benton et al., Reference Benton, Buster, Pinchuk, Glaws, King, Maclaurin and Chernyakhovskiy2024) and we use two variables: the $ u $ and $ v $ components of the wind velocity at 10 m from the surface for this task. We perform the following experiments with the ERA5 to WTK downscaling setup:

1. 1. Standard downscaling: We train and test all the neural operators and the baseline models with an upsampling factor of 5x, to go from 30-km to 6-km. The LR and the HR sizes are 53 × 53 and 265 × 265 for one region and 40 × 106 and 200 × 530 for the other region.

2. 2. Zero-shot downscaling: For the zero-shot setup, we use the models trained with the 5x upsampling and evaluate them on an upsampling factor of 15, going from 30-km to 2-km. While the LR sizes are the same as above, the HR sizes for the zero-shot case are 795 × 795 for one region and 600 × 1590 for the second region.

Dataset details We use the NREL’s WTK (Draxl et al., Reference Draxl, Clifton, Hodge and McCaa2015) as the ground truth dataset. WTK has a spatial resolution of 2-km and a temporal resolution of 1-hour. We get LR images from the ERA5 dataset (Hersbach et al., Reference Hersbach, Bell, Berrisford, Hirahara, Horányi, Muñoz-Sabater, Nicolas, Peubey, Radu, Schepers, Simmons, Soci, Abdalla, Abellan, Balsamo, Bechtold, Biavati, Bidlot, Bonavita, Chiara, Dahlgren, Dee, Diamantakis, Dragani, Flemming, Forbes, Geer, Haimberger, Healy, Hogan, Hólm, Janisková, Keeley, Laloyaux, Lopez, Lupu, Radnoti, Rosnay, Rozum, Vamborg, Villaume and Thépaut2020), available at 30-km (~0.28 degree) spatial resolution and 1-hour temporal resolution. This data has two channels, the $ u $ and $ v $ components of the wind velocity at 10 m from the surface. We have paired ERA5 and WTK datasets over two regions in the US, with image sizes ~800 × 800 and ~600 × 1600 respectively (see Figure 2 and (Benton et al., Reference Benton, Buster, Pinchuk, Glaws, King, Maclaurin and Chernyakhovskiy2024) for more details).

Figure 2.

Figure shows the two regions used for the ERA5 to WTK downscaling experiments. The 600 × 1600 (800 × 800) region is shown in black (blue). We use NREL’s rex (Rossol and Buster, Reference Rossol and Buster2021) tools to rasterize the WTK dataset.

Models are trained to map from coarse 30-km ERA5 to fine 6-km WTK with a 5x upsampling factor. This HR data is created by coarsening the WTK grid from 2-km to 6-km resolution. We realign, that is regrid 30-km ERA5 to the 6-km WTK coarsened grid using inverse distance weighted interpolation. For zero-shot experiments, we map the 30-km low-resolution ERA5 to the original 2-km resolution WTK (a 15x upsampling) for evaluation. The year 2007 is split 80/20 between the training and validation. We keep the year 2010 for testing. A single model is trained for both regions. While training, we ensure that every batch has an equal number of LR and HR pairs from both regions. We extract patches of size 160 × 160 from the WTK image tiles to obtain the HR images and the corresponding coarse patches (32 × 32) from ERA5 image tiles as the LR images for training (Benton et al., Reference Benton, Buster, Pinchuk, Glaws, King, Maclaurin and Chernyakhovskiy2024). This patch size is a hyperparameter tuned on the validation set. We normalize each channel separately with a mean and standard deviation before training. See Appendix A.1 for further details on the training process.

5. Evaluation

5.1. ERA5 to ERA5 downscaling

Error Metrics We show the results for the ERA5 to ERA5 standard downscaling experiments and zero-shot experiments in Table 1. Standard downscaling compares models trained and evaluated with an upsampling factor of 8x. We observe that SwinIR outperforms every other model in terms of MSE, MAE, IN, and PSNR. DCNO is a close second and the best-performing neural operator model. The DFNO model shows improved results over the vanilla FNO indicating the advantage of adding convolutional RRDB layers that learn spatial domain features useful for downscaling (as shown in ESRGAN (Wang et al., Reference Wang, Yu, Wu, Gu, Liu, Dong, Qiao and Loy2018)). Table 1 zero-shot results show a performance comparison between models trained on a 4x upsampling factor but evaluated zero-shot on generating 8x upsampled HR outputs. The zero-shot experiments show that the SwinIR is still the best-performing model. While DUNO is best among the neural operator models at zero-shot ERA5 to ERA5 downscaling, DCNO performs much worse than it did on standard downscaling. All the neural operator models are better than bicubic and SRCNN for both the standard and zero-shot downscaling. We also include downscaling results on the temperature and total column water vapor variables for the ERA5 to ERA5 experiments in Appendix C (Tables 6, 7) and observe similar relative model performance, with SwinIR outperforming every other model in both the standard and zero-shot downscaling in terms of all the metrics.

Table 1.

ERA5 to ERA5 wind speed downscaling results. MSE has units $ {\left(m/s\right)}^2 $ , MAE $ m/s $ and IN $ m/s $ . We bold the best-performing model among all the models and underline the best-performing neural operator model. Results for the other channels are added to the Appendix C

Takeaways: SwinIR outperforms every other model in both the standard and zero-shot downscaling in terms of all the metrics. While we expected neural operator models to be better, especially in the zero-shot experiments, they do not perform as well as SwinIR. DUNO achieves the best zero-shot downscaling performance amongst the neural operator models, while DCNO achieves much lower errors than DUNO in the standard downscaling setting.

Energy Spectrum Figure 4 shows zero-shot downscaled wind speed for the SwinIR, ESRGAN, DFNO, DUNO, DCNO, and DAFNO models, alongside the LR, HR, and bicubic interpolated images. We observe that SwinIR appears to learn fine-scale features closest to the HR or the ground truth image. We refer to the energy spectrum plots in Figures 3a and 3b to show the kinetic energy distributions as functions of wavenumber, across all the downscaling models. For the standard downscaling case in Figure 3a, ESRGAN matches the HR spectrum at all wavenumbers, DAFNO is second to ESRGAN at high wavenumbers, even though DAFNO is behind the DCNO model in terms of the error metrics. Figure 3b demonstrates the energy spectrum for the zero-shot downscaling scenario. SwinIR best captures the physical properties of the ground truth at low-to-medium wavenumbers, ESRGAN is better at medium-to-high wavenumbers but DAFNO is the best at the highest wavenumbers, even though they underestimate the energy content at higher wavenumbers. DCNO matches the HR curve for lower wavenumbers but falls behind ESRGAN at higher wavenumbers. We observe that SwinIR, ESRGAN, and EDSR produce peaks at the very high-end wavenumbers but the neural operator models except DAFNO do not introduce this high-end noise. Moreover, DUNO and DFNO seem to have similar intermediate peaks as bicubic interpolation, but that’s not the case with DAFNO and DCNO. We suggest this points to the difference in spatial domain features learned by DAFNO and DCNO which leads to a change in their spectrum. We also suspect the intermediate artifacts or spikes in ESRGAN and DAFNO (in both Figures 3a and 3b) might be caused by aliasing.

Figure 3.

Figures (a) and (b) show kinetic energy spectrum plots for ERA5 standard downscaling and zero-shot downscaling respectively. Kinetic Energy is normalized and wavenumber is measured relative to the domain size. We add a zoomed-in plot (right) beside the main plot to zoom in on the key region of interest. We observe that ESRGAN appears to be the best in capturing the ground truth energy spectra across all wavenumbers for standard downscaling (a) and at medium-to-high wavenumbers for zero-shot downscaling (b). DAFNO is the best performing at the highest wavenumbers in (b).

Figure 4.

ERA5 wind speed visualizations in $ m/s $ generated from the zero-shot downscaling. We zoom in on a small region for better comparison. SwinIR captures better and finer details of the HR image, over neural operator models, especially in the zoomed-in region. It is also better over regions with complex terrain (e.g. the mountain ranges in North and South America).

Takeaways: All the models underestimate the energy content at the higher wavenumbers, for the case of zero-shot downscaling. For the highest wavenumbers, or the dissipation range, this underestimation is most significant (for all the models except DAFNO). The inherent length dependence of the dissipation range makes zero-short downscaling a challenge for even the best-performing models. This is not surprising, as, for example, it is challenging for the models to fill in smaller-scale physical features if they do not see this level of detail when training on smaller upsampling factors. ESRGAN appears to be the best in capturing the ground truth energy spectra across all wavenumbers for standard downscaling and at medium-to-high wavenumbers for zero-shot downscaling. However, DAFNO outperforms all the models at the highest wavenumbers for the zero-shot downscaling case.

5.2. ERA5 to WTK downscaling

Error Metrics We show the results for the ERA5 to WTK downscaling in Table 2. As discussed in Section 4.2, using ERA5 as the LR for this setup makes it more challenging as the LR data is obtained from a different simulation than the HR data rather than using a coarsened version of the HR data. Table 2 shows the (1) standard downscaling results obtained from evaluating the models mapping ERA5 to WTK for a 5x upsampling factor and (2) zero-shot downscaling results where we evaluate the models trained on 5x upsampling to generate HR outputs at a 15x upsampling factor. SwinIR remains the best-performing model in both setups. DCNO achieves the best standard downscaling scores among the neural operators but is poor at zero-shot downscaling, as can be seen in Table 2 zero-shot results, where it performs worse than bicubic interpolation. We also observe DAFNO to be performing worse than bicubic interpolation at zero-shot downscaling. EDSR is a close second to SwinIR in both experiments. DUNO performs better than the other neural operators (and bicubic as well as SRCNN) at zero-shot downscaling.

Table 2.

ERA5 to WTK wind speed downscaling results. MSE has units $ {\left(m/s\right)}^2 $ , MAE $ m/s $ and IN $ m/s $ . We aggregate the error metrics over u and v wind velocity channels. We bold the best-performing model among all the models and underline the best-performing neural operator model

Takeaways: Overall, we observe the same relative model performance in this more difficult setting (ERA5 to WTK downscaling) as the easier setting (ERA5 to ERA5). The most significant takeaway remains that SwinIR is the best model at ERA5 to WTK zero-shot downscaling.

Energy Spectrum The energy spectrum plots for the ERA5 to WTK downscaling experiments are presented in Figures 5a and 5b. ESRGAN matches the HR spectrum at all wavenumbers for the standard downscaling. ESRGAN also comes closest to matching the HR energy spectrum for the zero-shot downscaling. However, as seen in Section 5.1, these models still underestimate the energy content in the high wavenumber range for zero-shot downscaling. SwinIR and EDSR follow behind ESRGAN for both setups, but they outperform all the neural operator-based models. We observe DCNO to be close to the HR curve for most of the wavenumbers except for the highest ones for standard downscaling (Figure 5a), but it is much worse at zero-shot downscaling (Figure 5b). DAFNO no longer shows good performance for this experimental setup. FNO performs the worst, consistent with their performance in terms of the average error metrics (as seen in Table 2). Figure 6 compares the zero-shot downscaled outputs in terms of the wind speed for the bicubic interpolation, ESRGAN, SwinIR, DFNO, DUNO, DCNO, and DAFNO models. We see fine-scale features of the ground truth captured better in ESRGAN’s downscaled outputs over other models.

Figure 5.

Figures (a) and (b) show kinetic energy spectrum plots for the ERA5 to WTK standard downscaling and zero-shot downscaling respectively. Kinetic Energy is normalized and wavenumber is measured relative to the domain size. We add a zoomed-in plot (right) beside the main plot to zoom in on the key region of interest. ESRGAN matches the HR spectrum at all wavenumbers for the standard downscaling (a). ESRGAN also comes closest to matching the ground truth energy spectrum for the zero-shot downscaling (b), but the gap increases for higher wavenumbers. SwinIR and EDSR rank second to ESRGAN for both (a) and (b).

Figure 6.

WTK wind speed visualization in $ m/s $ generated from the zero-shot downscaling (figure shows results on one of the two regions). We observe ESRGAN’s downscaled outputs (followed by SwinIR’s) to be sharper with better details than the neural operator models.

Takeaways: ESRGAN is the closest to the energy spectrum of the ground truth HR data, but, there is an underestimation in the energy content for the higher wavenumber range, in the zero-shot downscaling. None of the neural operator models, including DAFNO, perform well in capturing the HR energy spectrum.

5.3. Ablation study: Localized integral and differential kernel layers

In this section, we ablate using localized operator layers (Liu-Schiaffini et al., Reference Liu-Schiaffini, Berner, Bonev, Kurth, Azizzadenesheli and Anandkumar2024) in the FNO and FNO-based models. It has recently been shown that global operations in FNOs limit their ability to capture local features well (Liu-Schiaffini et al., Reference Liu-Schiaffini, Berner, Bonev, Kurth, Azizzadenesheli and Anandkumar2024). (Liu-Schiaffini et al., Reference Liu-Schiaffini, Berner, Bonev, Kurth, Azizzadenesheli and Anandkumar2024) propose two localized operators: differential operators and integral operators with local kernels, and augment FNO layers with them to show improved performance on PDE learning tasks. We add these localized operators into the FNO and FNO-based models (DFNO and DUNO) and compare them against the same models without the local layers. Inspired by the strength of SwinIR and EDSR, we explore whether incorporating local layers might help the FNO-based models learn better local features and improve their performance in our downscaling task.

Table 3 shows the results of our ablation study on the effect of local layers (Liu-Schiaffini et al., Reference Liu-Schiaffini, Berner, Bonev, Kurth, Azizzadenesheli and Anandkumar2024) on the FNO, DFNO, and DUNO modeling frameworks. Overall, the numbers do not indicate a significant impact on the MSE scores, especially for the ERA5 experiments. We observe FNO with the local layers to be better than without them on the ERA5 to WTK downscaling experiment, and DFNO and DUNO without the local layers to be better across most experiments. We suspect this might be pointing to the difference in architectures between the FNO and DFNO (or DUNO) models where the latter apply neural operators on the upsampled features obtained from RRDB layers. We note that the results in Sections 5.1 and 5.2 are for FNO with local layers and the DFNO and DUNO results are without them.

Table 3.

Ablation study on the effect of local layers from Liu-Schiaffini et al. (Liu-Schiaffini et al., Reference Liu-Schiaffini, Berner, Bonev, Kurth, Azizzadenesheli and Anandkumar2024) on FNO, DFNO, and DUNO. We show the MSE in all the experiment setups. The local layers mostly benefit FNOs for the downscaling tasks but do not improve the performance of DFNO or DUNO models