Back in the old days, local weather forecasting was performed by lore based on observed patterns. Many of these are sort of valid today – the most famous from England is ‘Red sky at night, sailor’s delight, red sky at morning, sailor’s warning.’ Other locations have their own, such as one used by the fishermen of the Aegean Coast of Turkey – ‘if you wake up to wet decks, the Meltemi won’t blow’ – which held true for the two years that I worked there!
After some terrible losses, Admiral FitzRoy (of HMS Beagle fame) created the first national weather centre – the Met Office – for the Royal Navy. He set up a vast network of observation posts throughout the British Isles and organised a storm warning system for each harbour of hoisting a red cone from a flagpole. He initiated the widespread use of accurate barometers and temperatures. The meticulous records from his observation posts are still found in their leatherbound journals in the Met Office library. His dream was to able to produce reliable weather forecasts. He committed suicide before this was realised, but the UK Met Office and its advances in meteorology are his legacy.
Weather forecasting has come a long way in the past hundred years or so. Just before the war a Norwegian called Bjerknes devised the cyclonic model for the weather. By considering the air masses, atmospheric pressure changes and the patterns in their motion, he used observations to windward to be able to, with reasonable accuracy, predict the next few day’s weather. He set in motion the system of standardised weather observations that are still in use today. More importantly, he also set up the international system of sharing observations and data. The rationale for this was that weather does not recognise political boundaries. Both of these concepts are still in use today and managed by the World Meteorological Organisation.
Around the same time, a young English mathematician called Lewis Fry Richardson became convinced that as weather follows the same laws of physics as everything else that it should be possible to predict the weather if a) the current state of the atmosphere is known and b) it was possible to calculate the future state based on the equations of fluid motion. He envisioned a concert hall full of mathematicians calculating each iteration for each section of the system using slide rules. To test this theory, he took all the weather observations for a single day. He spent years working through the calculations to produce a ‘forecast’ for the following day. As a forecast it was of course useless, as that day was long since history. But if his forecast matched the records from that day, his technique would be proved in concept. It wasn’t even close. His problem was that he just didn’t have enough observations to be able to confidently map out the initial state of the atmosphere. And it is next to impossible to gain an understanding of the weather without going backwards a few days to establish the patterns. In terms of science – it was like plotting the next point on a graph where you don’t know the trend, haven’t got enough data and don’t really understand what is happening. Given all these problems, Richardson put numerical weather forecasting on the backburner and went back to mathematics.
The big change happened with the advent of computers. Now there was a chance that the calculations could be done faster than the weather occured. Richardson’s dream of the hall of mathematicians has been replaced by enormous super computers. The most powerful computers in the world are dedicated to these calculations. The mathematics behind these calculations is called a WEATHER MODEL.
At the same time, the advent of satellites also drove the improvement of Bjerknes’ cyclonic model – it was now possible to see the patterns of the air masses and clouds from above.
Together, forecasters have dramatically improved weather forecasting. But there are still problems.
While every national weather agency uses the same input data that is routed through the WMO, each has programmed their models slightly differently. The basic physics is the same, but they vary the emphasis on certain factors and smooth the initial state data in different ways. This leads to variation in forecasts between models. While there is a natural tendency to want to produce the best model, it turns out that this variation in forecasts is surprisingly useful for the human forecasters in charge. At least until the ability is there to more accurately predict the future!
What is a Weather Model?
A mathematical model is a simplified representation of reality. A famous student example is Minecraft. In this case, the atmosphere (and the surface/ocean) is divided into small sections over many layers. It has to include vertical layers as air moves vertically as well as horizontally. At each edge, the calculations predict what the air at that point will do next. The Earth is very large, and even using a grid spacing of 20 km and vertical intervals of a few hundred metres, at a time interval of 10 minutes, the number of calculations is enormous. In order to perform that many calculations at a rate that will produce a useful forecast before the event, vast supercomputers are needed.
In theory, as the grid spacing (resolution) and the time intervals get ever smaller the forecast accuracy will continue to improve.
How National Weather Forecasts are Produced
The biggest difference between the forecasts produced by a national weather service (e.g. Bermuda Weather Service BWS) and those produced by an app (e.g. Windguru or Apple Weather) is that the human forecaster is still a critical player. The automatic weather forecasts essentially interpolate the data from a particular numerical model (often the US Global Model, called the GFS) and repackage it for their users. Some of these are really presented well. In particular, www.windy.com shows the atmospheric patterns well from two different models. The resolution is still over 25 km though, but looks finer.
The numerical weather model forecasts take time to run. First they need to assimilate the observational data from around the world and jiggle the previous forecast to match these observations to produce an ‘analysis’. This is a representation of how the atmosphere is at the point of the global observations. Hopefully, it is not too different from the previous forecast. Differences represent the 24 hr forecast errors. Some data is weighted heavier than others. Then, it has to be run forward in time to produce the forecast for at least the next 24 hrs ahead. It may be that the forecast takes several hours to produce. In that time, the weather may be a bit different to the model.
The human forecasters at BWS are constantly comparing the forecasts from a variety of models with the latest observations (satellite, radar, out-of-the-window, aircraft and shipping) and determine which model is working best for that day. If all the models agree, then the forecasters have a lot of confidence in their forecasts. If they don’t agree, then the forecast confidence is low. Forecasters still use the cyclonic model and discussions of air masses. It is useful, and sometimes entertaining, to read the forecaster’s discussions.
The image above shows the numerical weather forecasts from two models; the ECMWF (European Centre for Medium Range Weather Forecasts) and the US National Weather Service’s GFS model. Both show a low pressure system north of Bermuda, with a cold front about to sweep across the island. The models are pretty close, but there are some differences in the location of the depression and the exact timing of the cold front hitting the island – the GFS has the front arriving slightly earlier than the ECMWF’s model (by the way, I can’t remember which one got it right!) The forecasters at BWS would be comparing these, along with other models from the Met Office and MeteoFrance etc with the radar and satellite imagery. The forecast that matches reality best (termed nowcasting) is the one that they will base their next forecast on.
The swipe image below shows divergence in model forecasts. The lead time is some seven days, which is stretching the limits of forecast accuracy. Compare the forecasts for Bermuda between the ECMWF and the GFS models – very different. ECMWF has SW winds with a cold front coming down from the north, while the GFS model has the front go through much earlier and northerly winds all day. Which is the best forecast? Only time will tell.
Ensemble Forecasting
The three main sources of errors in numerical weather predictions are caused by:
- incomplete global observations – even now, there are still not enough data points covering the world!
- model resolution – limited by computing power
- Incomplete understanding of the physics – this leads to approximations in some cases
A error in building an accurate analysis will lead to a domino of errors with the forecast, especially after more than three days ahead. This is sometimes known as the Butterfly Effect, after the story that a butterfly flapping its wings in the forest can cause a storm halfway around the world.
Ensemble forecasting is done in two ways. The first is to compare the outputs from all of the various global models. The other can be done by each individual agency. What they can do is to run low resolution versions of their models with slight variations of the starting analysis. If the output of most of these model runs leads to a similar result, then they have a high confidence in what the atmosphere is going to do. If they vary wildly, then they have little confidence in their forecasts. The most well-known example of ensemble forecasting is for hurricane prediction – the so-called ‘spaghetti plots’.
A spaghetti plot (ensemble forecast) for the track of Tropical Storm Jerry in September 2019. Most of the models follow the same path for the next couple of days, then after that they start to diverge. This shows that they had a high level of confidence for the next 72 hrs, then after that the confidence decreases.