research impact improving weather forecasts to avert disruptions damage disaster

Improving weather forecasts to avert disruptions, damage, and disaster – clean

/in /by

Prof. Smith and his group’s contribution in this area continued throughout the years, and the systems operational today are still benefitting from his research done years ago.

– A senior official in the European Centre for Medium-Range Weather Forecasts

Impact Case Study:
Improving weather forecasts to avert disruptions, damage, and disaster

Research by Professor Leonard Smith, Centre for the Analysis of Time Series

PDF

LSE research has helped government and industry to develop ways to better use weather forecasts and improve contingency planning

 What was the problem?

Weather has an impact on people, businesses and economies every single day. Even fairly common weather, with no extreme meteorological elements, can produce costly disruptions to the way families, communities and societies function.

Problems can include changes in demand and disruptions to power and water supplies, the production and distribution of food and other basic needs, travel systems, and communication networks such as the Global Positioning Satellite. Even day to day changes in forecasts themselves can have huge impacts.

There is therefore a strong need for more informative weather predictions which enable people with different needs and levels of understanding to plan for and manage their responses to changing weather conditions and to changes in their degree of confidence in the forecast itself.

What did we do?

A team of researchers at the LSE Centre for the Analysis of Time Series (CATS), led by Professor of Statistics Leonard Smith, looked at ways of improving traditional weather forecasting.  They interpreted modern  “ensembles” of forecasts, as opposed to the traditional single forecast, to better represent today’s forward view of what is essentially an uncertain and chaotic system.

Because weather models are imperfect, there are some days when even the ensemble forecasts appear, in hindsight, not to have been accurate. This too is taken into account. The approach is to consider an ensemble as merely a source of information rather than a collection of equally likely scenarios. Smith demonstrated that the European Centre for Medium-Range Weather Forecast’s (ECMWF) 51 individual forecast points could be used to produce information regarding the reliability of a single forecast or, alternatively, a probability forecast of a combination of several weather variables. He then looked at developing better ways of communicating both the forecasts of the weather and the probability of those particular forecasts.

“Weather Roulette” – which was developed to literally assess the claim ‘I bet I can do better than your forecast’ – is a framework for evaluating the performance of forecasts by translating the probabilities into effective daily interest rates (a measure used in finance to show the actual return from a particular interest rate). The exercise starts with an initial investment, such as one euro, and after every round of betting on the weather outcome, the total value (profit and initial value) is reinvested.

Weather Roulette offers a new approach for communicating the performance and relative value of probabilistic forecast systems. It provides a simple and easy-to-understand framework for evaluating decisions based upon probabilistic weather forecasts. It makes it easier to communicate the results of weather forecasts and can also be used to evaluate the relative performance of different forecasts and the economic value of the forecasts.

Ensembles are particularly useful in situations where small probabilities can lead to large costs or benefits. People with financial skills, such as energy traders and weather centre managers, can easily grasp Weather Roulette and apply it in their own practice-based environments.

What happened?

Official weather forecasters

Smith and CATS have developed significant and long-standing relationships with a number of institutions involved in weather forecasting, including:

  • The European Centre for Medium-Range Weather Forecasts (ECMWF), where Smith’s work helped them to refine their techniques and methodology
  • the US Naval Research Laboratory, where a group is forecasting hurricanes in the Pacific
  • the US National Centre for Atmospheric Research, where Dr Hailiang Du, an LSE postdoctoral research officer, is sharing methods and providing a test bed for data assimilations and where observations are being incorporated into a computer model of a real system
  • the International Research Institute for Climate and Society and the World Climate Research Programming Working Group on Regional Climate.

Since 2010 ECMWF has incorporated Weather Roulette into its Ensemble Verification Training Course. Weather Roulette is also an essential component of a ‘training of trainers’ course that has been run by the World Meteorology Organisation’s Commission on Climatology over the past five years with representatives from regional climate centres around the world. As of 2014, 150 trainers from seven countries from as far afield as China and South Africa had gone through the training and learned how to use Weather Roulette.

The World Meteorology Organisation has included Weather Roulette as one of its recommended procedures in its Guidance on Verification of Seasonal Climate Forecasts, which has been officially disseminated to its 191 member countries and territories.

The United Kingdom Met Office has considered Smith’s research in the redesign its 3-month Outlook product, which provides an indication of possible average temperature and rainfall conditions over the UK for the upcoming three months. The Cabinet Office makes this product available on an ongoing basis to managers in both the public and private sectors to enable them to make contingency plans for extreme weather events and potential emergencies.

Impact on industry

Smith and CATS also have a long history of partnership with industry in embedding their research findings in contexts where real-time forecasting is helpful in managing uncertainty and risk. They have focused significant effort on constructing actionable weather information from forecasts on which specific users can take practical decisions. Specific applications have included: road gritting; food sales; horse race-track conditions; crop forecasting; insurance brokerage; energy trading; wind power production; hydrocarbon exploration and production (oil reservoirs and flows); and electricity generation.

Around a third of Smith’s students and post-docs are applying this knowledge as employees of organisations, including the Bank of England, Royal Dutch Shell, the Met Office and Risk Management Solutions. Current CATS partnerships are also underway with the UK Department of Energy and Climate Change (DECC) and the Royal National Lifeboat Institute (RNLI).

CATS researchers have been working with the Royal National Lifeboat Institute to identify the weather conditions that tend to cause a high rate of incidents and those that are more likely to result in serious or life-threatening incidents. This work will enable the Institute to make informed planning decisions, such as the positioning of crew or the best time to perform maintenance on the station’s equipment.

Smith has worked with EDF Energy to develop methods for forecasting electricity demand, which has helped to reduce risk, manage supply and improve performance.

Metra, the global commercial arm of the New Zealand meteorological service, worked with CATS on the design and marketing of a product called Vantage. This helped energy managers and traders to manage weather-related opportunities and risks and make operational decisions. It led to a new generation of Metra products and services in use worldwide.

research impact ensuring the best science based predictions of climate change

Ensuring the best science-based predictions of climate change – clean

/in /by

Professor Smith understands better than most climate scientists what the limitations of the science are and how to use statistical and physical analysis to draw robust conclusions for policy makers.

-Dr James Baker
Co-lead author of the Harvard Report for the Central Intelligence Agency

Impact Case Study:
Ensuring the best science-based predictions of climate change

PDF

Research by
Professor Leonard Smith
Centre for the Analysis of Time Series

LSE researchers have challenged the validity of current models used to predict climate change and pressed for credible science-based policymaking.

What was the problem?

Climate change is arguably the greatest threat that humans face in this century. Scientists from many disciplines are involved in defining the exact nature of this threat, evaluating its magnitude and identifying the most effective ways to respond to it.

An important aspect of this work is producing models that simulate possible future scenarios and assess their probabilities. Such scenarios enable policymakers and planners to explore and make decisions on the nature and timing of individual and collective responses.

Unfortunately, the mathematical models on which such decisions have been based are imperfect. Interpreting the outputs of models in each particular case requires estimating the extent to which one can interpret the simulations as high fidelity reflections of the future.

Given multiple factors that interact with one another in highly dynamic ways, the future state of the climate system is expected to evolve in a non-linear fashion and reach states that are qualitatively different from what we have experienced to date. At stake is how we adapt to and plan for climate change. Over-interpreting projections of climate probabilities can produce poor policy choices and leave us vulnerable to more intense and devastating forms of change.

Also at stake is the credibility of science-based policymaking.

What did we do?

Climate simulation research within LSE has taken place within a broader research programme on nonlinear dynamical systems carried out by the Centre for Analysis of Time Series (CATS) over the last decade.

This research has challenged the fidelity of current climate-change models and their projections of future climate states. The research has been led by Professor Leonard Smith and Dr David Stainforth, aided by graduate researchers E. R. Tredger, Ana Lopez and Erica Thompson.

Mathematical research in 2004 by Smith and CATS Visiting Fellow Kevin Judd had identified the consequences of model imperfections when forecasting probabilities from ‘noisy’ observations and models of chaotic systems.

These findings were then applied specifically in the context of climate models. In 2005 CATS researchers sought to interpret the largest ensemble of relevant climate simulations generated by climateprediction.net, a project co-founded by Stainforth for which Smith was co-investigator and a key player in its conceptual design.

Their work explored a wider range of variety of model behaviour than ever seen before, demonstrated that simple, reasonable looking statistical approximations failed in practice, and also suggested that model-based probability distributions have limited fidelity in terms of their predictive value extrapolating into the far future. Each generation of models has a limited range after which the information and numbers they generate are not expected to reflect the future. Any ‘probabilities’ produced by the models beyond that range are of highly limited relevance for cost-benefit analyses of policy options, and may well prove maladaptive.

What happened?

Since 2002 Smith’s research has stimulated and informed policy debate both nationally and internationally. CATS’ conclusions have been noted by the world’s leading scientific body on climate change, the United Nations Intergovernmental Panel on Climate Change. The Panel’s Fourth Assessment Report cites Smith as the authority in terms of noting the significant impact of model inadequacy on its ‘probability’ distributions.

Within the UK, Smith and Stainforth’s improved interpretation of climate-model simulations has contributed to changes in the way climate projections are presented and has influenced government policy on climate change more generally. More specifically, their work affected the government’s first Climate Change Risk Assessment of 2012, prepared in response to the Climate Change Act 2008, which committed the government to significant reductions in greenhouse gas emissions.

Smith and Stainforth remain vocal and engaged consultants regarding the interpretation and use of climate projections, particularly the UK Climate Projections 2009 (UKCP09), which are still current. These contain projections of future changes to the UK climate until the end of the century at very very high resolution.

At the request of the Department for Environment, Food and Rural Affairs, Smith and Stainforth were present at the initial meeting called to agree the methodology for the Assessment. They objected strongly to the intense pressure to over-interpret the output of climate models.

National meteorological services often wish to avoid known defects of probabilities-based approaches.  For example, Smith and Stainforth worked with the Dutch meteorological office and Dutch government scientists in their successful efforts to introduce an alternative approach to climate risk management. More recently, the Dutch government invited members of CATS to a closed-door meeting on the presentation of uncertainty in the 2013 report of the UN Intergovernmental Panel on Climate Change.

Smith and Stainforth, together with Roman Frigg of the Philosophy Department and a co-director of CATS, continue to engage with government departments and the now independent UK Climate Impact Programme to improve the quality and coherence of information relating to climate change.

Commerce and industry have also turned to CATS for help and have modified their approaches to climate risk management as a direct result of understanding CATS’ research. Retained by major companies in the energy sector, Professor Smith has reviewed UK Meteorological Office commercial projects targeting the energy infrastructure under future climate change as a representative of the energy sector. His continued engagement with insurance industry partners such as Munich Re and Lloyd’s of London has also enabled these companies to interpret climate information more effectively.

Internationally, Professor Smith was a key participant in a series of meetings and reviews that produced the Harvard Report, Climate Extremes: Recent Trends with Implications for National Security (2013). Funded by the US Central Intelligence Agency, the study explored the forces driving extreme weather events and their impacts over the next decade, concentrating on their implications for national security plans. Smith’s contribution focused on the extent to which the risks of climate change can be quantified, and he is named as one of the report’s major reviewers. He has also regularly engaged on climate-change issues with the offices of individual US Senators and Congressmen and with US scientific associations such as the American Statistical Association.