We present a deep learning architecture that reconstructs a source of data at given spatio-temporal coordinates using other sources. The model can be applied to multiple sources in a broad sense: the number of sources may vary between samples, the sources can differ in dimensionality and sizes, and cover distinct geographical areas at irregular time intervals. The network takes as input a set of sources that each include values (e.g., the pixels for two-dimensional sources), spatio-temporal coordinates, and source characteristics. The model is based on the Vision Transformer, but separately embeds the values and coordinates and uses the embedded coordinates as relative positional embedding in the computation of the attention. To limit the cost of computing the attention between many sources, we employ a multi-source factorized attention mechanism, introducing an anchor-points-based cross-source attention block. We name the architecture MoTiF (multi-source transformer via factorized attention). We present a self-supervised setting to train the network, in which one source chosen randomly is masked and the model is tasked to reconstruct it from the other sources. We test this self-supervised task on tropical cyclone (TC) remote-sensing images, ERA5 states, and best-track data. We show that the model is able to perform TC ERA5 fields and wind intensity forecasting from multiple sources, and that using more sources leads to an improvement in forecasting accuracy.

In order to enable widespread integration of solar energy into the power system, there is an increasing need to reduce the uncertainty associated with solar power output which requires major improvements in solar irradiance forecasting. While most recent works have addressed short-term (minutes or hours ahead) forecasting, through this work, we propose using deep sequence learning models for forecasting at longer lead times such as a week in advance, as this can play a significant role in future power system storage applications. Along with point forecasts, we also produce uncertainty estimates through probabilistic prediction and showcase the potential of our machine learning frameworks for a new and important application of longer lead time forecasting in this domain. Our study on the SURFRAD data over seven US cities compares various deep sequence models and the results are encouraging, demonstrating their superior performance against most benchmarks from the literature and a current machine learning-based probabilistic prediction baseline (previously applied to short-term solar forecasting).