|  | | IBM Research | |
Precision Forecasting for Weather-Sensitive Business Operations
"Deep Thunder"
is a service that provides local, high-resolution weather predictions customized to
business applications for weather-sensitive operations up to a day ahead of time.
In particular, the goal is to provide weather forecasts at a level of precision and fast enough
to address specific business problems. Such forecasts can be used for competitive advantage
or to improve operational efficiency and safety. Users want access to the forecasts in
an "on-demand" fashion -- disseminated to them the way they want them, when they want them.
In reality, improving the effectiveness of a customer's weather-sensitive operations is not
really about the weather. Rather, it is one of optimization of business processes such
as resource allocation, scheduling and routing, which are constrained by specific weather
events. Long-term, the real value is when the forecasts are integrated into business
processes. Having detailed forecasts of the right caliber as a service is a critical
prerequisite to enable optimization of weather-sensitive operations.
In addition to the information on this page,
there is a shorter discussion of Deep Thunder.
In many applications, expected local weather conditions during the next day or two are critical factors in planning
operations and making effective decisions. Weather-sensitive business operations are often reactive to
short-term (3 to 36 hours), local conditions (city, county, state) due to unavailability of appropriate predicted
data at this temporal and spatial scale. Typically, what optimization that is applied to these processes to
enable proactive efforts utilize either historical weather data as a predictor of trends or the results of
continental-scale weather models. Neither source of information is appropriately matched to the temporal
or spatial scale of such operations. While near-real-time assessment of observations of current weather
conditions may have the appropriate geographic locality, but by its very nature is only directly suitable for
reactive response. Alternatively, cloud-scale (meso-gamma-scale) numerical weather models operating at
higher resolution in space and time with more detailed physics may offer greater precision and accuracy within
a limited geographic region for problems with short-term weather sensitivity. Such forecasts can be used
for competitive advantage or to improve operational efficiency and safety. There are many applications for
these highly focused weather forecasts in specific industries, whose decision making and operations are weather-sensitive
such as local government, transportation, agriculture, broadcast, energy, insurance, etc. each with specific societal
and economic benefits. To evaluate this hypothesis, the
IBM Thomas J. Watson Research Center initiated the
"Deep Thunder"
project focused on applications of local, high-resolution, short-term weather forecasting.
In particular, a prototype, operational system was initially built to provide
24-hour forecasts for the New York City area at 1 km resolution, which are updated twice daily. It also produces
forecasts at 4 km resolution, which cover the greater Tri-State region and beyond, and then 16 km resolution for
the northeast US. That capability was extended to include other parts of the United States.
An example appears to the right, which shows a prediction of Hurricane Wilma in southern Florida
produced by Deep Thunder.
While efforts are on-going to continue to refine the forecasts for the New York area since the first forecasts were
produced in 2001, they have been extended to provide forecasts for six additional metropolitan areas
in other parts of the United States, including the Miami/Fort-Lauderdale area.
These model-based forecasts provide detailed four-
dimensional information about temperature, winds, precipitation, etc.
from the surface of the earth to an altitude of about 16 km. This meteorological modelling effort is unique
in the industry and the academic community. The concept behind Deep Thunder is clearly to
be complementary to what the National Weather Service (NWS) does and to leverage their investment in making data,
both observations and models, available. The idea, however, is to have highly focused modelling by
geography and application with a greater level of precision and detail than what NWS provides
while addressing the needs of specific industries. Part of the effort is to improve the technology and
part is to understand the business, safety and other value that such modelling can provide.
In reality, improving the effectiveness of weather-sensitive operations is not really about the weather.
Rather, it is one of optimization of business processes such as resource allocation, scheduling and routing, which
are constrained by specific weather events.
The effort began with building a capability sufficient for operational use. In particular, the goal is to provide
weather forecasts at a level of precision and fast enough to address specific business problems. Hence,
the focus has been on high-performance computing, visualization, and automation while designing, evaluating and
optimizing an integrated system that includes receiving and processing data, modelling, and post-processing analysis
and dissemination. Part of the rationale for this focus is practicality. Given the time-critical nature
of weather-sensitive business decisions, if the weather prediction can not be completed fast enough, then it
has no value. Such predictive simulations need to be completed at least an order of magnitude faster than
real-time. But rapid computation is insufficient if the results can not be easily and quickly utilized.
Thus, a variety of fixed and highly interactive flexible visualizations focused on the applications have also
been implemented to enable timely use and assessment of the model forecasts.
Additional material is available for you to learn more about the Deep Thunder project. Some
information is below as well as via the links to the upper left.
In addition, a
video is available for
you to see, which was made early in this effort. It outlines some of the initial ideas behind
the Deep Thunder project. It will help introduce the work to you as
well as the material below and the links to additional details and examples.
You Can Learn More About Deep
Thunder
|
Given that weather-sensitive business operations are primarily reactive to short-term (3 to 36 hours),
local conditions (city, county, state) due to unavailability of appropriate predicted data at this
temporal and spatial scale, there is a need to improve the quality of local forecasts. One solution to this
problem is the introduction of additional tools and data focused on the region of interest. Consider
numerical weather prediction models that are typically run at relatively low or moderate resolution over a large
geographic region, that is, at a synoptic to a meso-beta scale. Meteorologists will employ such a model, their knowledge of
the region in question and local conditions derived from observations to arrive at a final forecast. The resolution
at which these models usually operate is often too coarse for predictions of local phenomena like thunderstorms and other
severe weather, wind shear, land-sea breezes, etc. However, forecasts can be substantially improved with the
application of regional and local numerical modeling techniques. These mesoscale models operate at higher resolution
and incorporate explicit cloud microphysics. They do not replace the broader scale simulations, but supplement
them by using the results to establish boundary conditions. To enable timely execution of these models, which is
required for operations, the simulation must be parallelized on a high-performance computing system. For this effort, an
IBM pSeries Cluster 1600* is
employed, which is a distributed memory MIMD computer. It is one type of supercomputer technology
available from IBM. Depending on the size and resolution of the domain(s) over
which forecasts are being produced and the configuration of the supercomputer system,
the continual generation of new 24-hour forecasts on
an update cycle every few hours can easily be supported. Many organizations use this same type of
computing system for weather modelling.
This simulation capability requires a change in how
one utilizes the results for the formulation or interpretation of a forecast.
Since large volumes of complex data are produced, the use of traditional graphical
representations of data for forecasters and decision makers can be burdensome. Instead of static or simple flip-book animations of
two-dimensional techniques like contour maps, novel three-dimensional visualization strategies are employed. An
example of such a visualization is at the beginning of this page. These methods are developed
from a perspective of understanding how the weather forecasts are to be used in order to create task-specific
designs.
From there, the requirements for the local physics and geography of importance can be determined that define how the
weather model should run in order to enable the visualizations.
In many cases, a "natural" coordinate system is used to provide a context for three-dimensional analysis,
viewing and interaction. They provide representations of the state of the atmosphere, registered with relevant
terrain and political boundary maps. Visualizations such as these also facilitate the dissemination of the computed
weather forecasts to the public via the World Wide Web as well as broadcast and print media.
The initial focus of Deep Thunder was on general forecasting but with a view toward
applications. These include travel, aviation,
agriculture, broadcast, communications, energy, insurance, sports, entertainment, tourism, construction and other
industries where weather is an important factor in making effective business decisions. This activity is part of
the on-going IBM initiative in Deep Computing -- to provide
the ability to analyze large amounts of data and develop solutions to very complex problems. Essentially, a further
goal of Deep Thunder is to enable proactive decision making affected by weather by coupling predictive
weather simulations to business processes, analyses and models.
One might ask what is the potential business
value of improved weather forecasts? As a start, consider the fact that
"weather is not just an environmental issue, it is a major economic factor. At least one trillion dollars of our
economy is weather sensitive," Former US Commerce Secretary
William Daley.
A more recent study reported in the
Bulletin
of the American Meteorological Society estimates that one third
of private industry activities representing some three trillion dollars
annually has some degree of weather and climate risk. A
partial summary of the market segment economic impacts is available. How these factors
relate to market opportunities has been
summarized by the
National Oceanic and Atmospheric Administration (NOAA),
the parent agency of the National Weather Service.
According to the
NOAA,
during the period from 1980 through 2005, the U.S. sustained over $390 billion in overall inflation-adjusted
damages/costs due to just extreme weather
events (i.e., over $1 billion in damage per event).
For example, the costs associated with weather-related emergency planning
and disaster warning average about $15.8B and 1500 lives in the US each
year. From 1988 through mid-1999, 36 major disasters in the US cost
over $170B. In 1998, worldwide damage related to weather was about
$92B.
However, these costs are across a wide range of
geographic and time scales.
Therefore, consider the more local and short-term impact of weather events.
For example, it has been estimated that the annual cost of under or overpredicting
electricity demand due to poor weather forecasts is several hundred million
dollars in the US alone. The value for weather forecast services for U.S.
households in 2001 was estimated at $11.4B. According to the Air
Transport Association air traffic delays caused by weather were about $4.2B
in 2000, of which $1.3B was estimated to be avoidable.
According to the United States Department of Transportation, about 7000 people are killed and
800,000 are injured each year in weather-related accidents on US highways. The economic impact of
these and other weather-related problems on the roads are estimated to lead to 544M vehicle-hours of delay,
and an economic impact of about $42B annually.
In addition, there is an emerging industry for weather derivatives
(as hedges against weather-related financial risk), which has grown from
nothing in 1997 to tens of billions of dollars today. Initially, this market was for
energy-related commodities, but has expanded to other markets like agriculture and retail. While
it focuses primarily on the seasonal scale, it may evolve to include the dynamics of the
short-term market, as the local impact of energy commodities grows.
A summary of these and other statistics is available from
the US Government.
Another common question about Deep Thunder is how is it different than the National Weather Service? First of all, it would not be possible without the
National Weather Service (NWS). Deep Thunder leverages the US Government's significant investment in
observing the atmosphere and simulating the weather by using the data that NWS makes available. NWS focuses on
uniform services for the whole US by providing detailed observations (spacecraft, radar, stations, etc.) and
global to continental-scale simulations on a large IBM pSeries Cluster 1600 (e.g., 12 km for North America), for example,
which are considerable tasks. Their mission does not include highly customized, detailed services for
business or other users. Complementary to NWS, Deep Thunder provides local, meso-gamma-scale,
high-resolution (e.g., 100 x 100 km at 1 km) simulations (on a small IBM supercomputer). Such forecasts can provide greater
precision and accuracy by incorporating more detailed physics and then can be customized to enable focused operations,
products and integration with businesses.
For further information on this system beyond what is discussed on this site, please contact either Tony Praino or Lloyd Treinish at IBM Thomas J. Watson Research Center,
or Don Stremme of IBM Sales and Distribution.
Deep Thunder is running on a regular, operational basis at the IBM Thomas J. Watson
Research Center for several metropolitan areas in the United States (typically, two 24-hour forecasts each day). The initial testbed
was for the New York City metropolitan area .
Sample results of triply
nested forecasts at 16, 4 and 1 km resolution are available for viewing. Similar nested
forecasts to 2 km resolution are now being produced operationally for the Chicago, Kansas City, Atlanta, Baltimore and Washington
metropolitan areas. In addition, experimental forecasts to 1 km resolution are being done for the
San Diego area, to 1.5 km resolution for the Miami-Fort Lauderdale area, and
2 km resolution for the greater Atlanta area.
The image below places all but one of these forecasts in a geographic context, which shows a
map of the eastern two-thirds of the continental United States. On the map
are three regions associated with six of the seven aforementioned metropolitan
areas. They correspond to the triply nested, multiple resolution forecasting
domains used to produce each high-resolution weather forecast.
The outer nests are in gray, the intermediate nests are in magenta and
the inner nests are in white.
The areas within the gray borders are covered at 32 km resolution for Kansas City,
Chicago, Atlanta and Baltimore/Washington, 24 km for Miami-Fort Lauderdale and 16 km resolution for New York.
The areas within the magenta borders are covered at 8 km resolution for Kansas City,
Chicago, Atlanta and Baltimore/Washington, 6 km for Miami-Fort Lauderdale and 4 km resolution for New York.
The areas within the white borders are covered at 2 km resolution for Kansas City,
Chicago Atlanta and Baltimore/Washington, 1.5 km for Miami-Fort Lauderdale and 1 km resolution for New York.
If you are interested in seeing the current results for any of these forecasts or discussing the
potential applications of Deep Thunder, please contact us
or visit IBM alphaWorks Services for
limited, real-time access to some forecasts generated by the Deep Thunder service.
lloydt@us.ibm.com Last
updated November 14, 2006
*Trademarks of the IBM Corporation in the United States
or other countries or both. | | | | |
