IBM, NASA Launch Open-Source AI for Weather, Climate on Hugging Face

New AI Foundation Model Offers Deeper Insights for Weather and Climate Analysis

September 23, 2024

YORKTOWN HEIGHTS, N.Y., September 23, 2024 /PRNewswire/ — IBM (NYSE: IBM) has introduced a groundbreaking AI foundation model designed to enhance the understanding and analysis of weather and climate data. This new model, developed in collaboration with NASA and with contributions from Oak Ridge National Laboratory, is available in open-source format to scientists, developers, and businesses. The model provides a flexible and scalable solution to address a variety of challenges related to both short-term weather forecasting and long-term climate projections.

The unique design and training regime of this model allow it to tackle a broader range of applications than existing weather AI models. According to a recent paper published on arXiv titled “Prithvi WxC: Foundation Model for Weather and Climate,” the potential applications include creating targeted forecasts based on local observations, detecting and predicting severe weather patterns, improving the spatial resolution of global climate simulations, and refining how physical processes are represented in numerical weather and climate models. In one experiment, the foundation model accurately reconstructed global surface temperatures from a random sample of only five percent of the original data, suggesting its broader application in data assimilation problems.

This model has been pre-trained on 40 years of Earth observation data from NASA’s Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2). Its architecture allows it to be fine-tuned to various scales—global, regional, and local—making it highly versatile for a range of weather studies.

Key Features and Applications

The foundation model is available for download on Hugging Face, along with two fine-tuned versions that address specific scientific and industry-relevant applications:

Climate and Weather Data Downscaling

Downscaling is a common meteorological practice that involves inferring high-resolution outputs from low-resolution variables. Typical data inputs include temperature, precipitation, and surface winds, which can have varied resolutions. This model can depict both weather and climate data at up to 12x resolution, generating localized forecasts and climate projections. The fine-tuned downscaling model is available on the IBM Granite page on Hugging Face.

Gravity Wave Parameterization

Gravity waves are prevalent throughout the atmosphere and can affect many atmospheric processes related to climate and weather, such as cloud formation and aircraft turbulence. Existing numerical climate models have traditionally not captured gravity waves accurately, leading to uncertainties in understanding their impact on climate processes. This foundation model can help scientists better estimate gravity wave generation, thereby improving the accuracy of numerical weather and climate models and reducing uncertainty when simulating future weather and climate events. The gravity wave parameterization model is available as part of the NASA-IBM Prithvi family of models on Hugging Face.

Statements from Key Players

“Advancing NASA’s Earth science for the benefit of humanity means delivering actionable science in ways that are useful to people, organizations, and communities. The rapid changes we’re witnessing on our home planet demand this strategy to meet the urgency of the moment,” said Karen St. Germain, director of the Earth Science Division of NASA’s Science Mission Directorate. “The NASA foundation model will help us produce a tool that people can use: weather, seasonal, and climate projections to help inform decisions on how to prepare, respond, and mitigate.”

“This space has seen the emergence of large AI models that focus on fixed datasets and single use cases—primarily forecasting. We have designed our weather and climate foundation model to go beyond such limitations so that it can be tuned to a variety of inputs and uses,” said Juan Bernabe-Moreno, Director of IBM Research Europe and IBM’s Accelerated Discovery Lead for Climate and Sustainability. “For example, the model can run both on the entire earth as well as in a local context. With such flexibility on the technology side, this model is well-suited to help us understand meteorological phenomena such as hurricanes or atmospheric rivers, reason about future potential climate risks by increasing the resolution of climate models, and inform our understanding of imminent severe weather events.”

“As a premier research institution and computing facility, we’re focused on supporting teams to make research breakthroughs across many areas of science,” said Arjun Shankar, director of the National Center for Computational Sciences at Oak Ridge National Laboratory. “Our collaboration with IBM and NASA to support the creation of the Prithvi weather and climate foundation model was a key part of our goal to bring advanced computing and data to problems of national importance, in this case, for weather and climate applications, which need continued computational science and model skill improvements to be impactful.”

Additional Collaborations and Future Prospects

IBM has already partnered with Environment and Climate Change Canada (ECCC) to test the model’s flexibility with additional weather forecasting use cases. ECCC is exploring very short-term precipitation forecasts using a technique called precipitation nowcasting, which ingests real-time radar data as input. The team is also testing the downscaling approach from global model forecasts at 15 km to km-scale resolution.

This weather and climate model is part of a broader collaboration between IBM Research and NASA to leverage AI technology for planetary exploration. It joins the Prithvi family of AI foundation models. Last year, IBM and NASA released the Prithvi geospatial AI foundation model, the largest open-source geospatial AI model available on Hugging Face. This geospatial foundation model has since been used by governments, companies, and public institutions to examine changes in disaster patterns, biodiversity, land use, and other geophysical processes. The foundation model and the gravity wave parameterization model can be accessed through the NASA-IBM Hugging Face page, and the downscaling model can be accessed through the IBM Granite Hugging Face page.

About IBM

IBM is a leading provider of global hybrid cloud and AI, and consulting expertise. We assist clients in over 175 countries in harnessing insights from their data, streamlining business processes, reducing costs, and gaining a competitive edge. Thousands of
For more Information, Refer to this article.