Weather forecast accuracy

So, you go on the web and everyone's doing 5 day (or longer) weather forecasts. But how accurate are these forecasts? I decided to do a little research and check myself. Here are the results.

Method

I picked a random 2 week period in January 2011 and each day noted the weather forecast for that day and the following 4 days and compared it with the actual weather. The 2 week period was random in that one day I simply decided to do it, and it lasted 2 weeks as after 2 weeks I went away so I couldn't check the actual weather. The location chosen to test the forecast accuracy was Newbury in Berkshire, UK.

Each morning at ~7am I noted the weather forecast for that day and the following 4 days. This was done by typing in my postcode and getting the forecast. I got this from the BBC website. I also tried the Met Office forecast and Metcheck. The BBC forecast seemed to be identical to the Met Office so after a couple of days I never bothered checking the Met Office site as the BBC's was easier to use. Metcheck only lasted 1 day as on day 2 their servers were down so I couldn't get a forecast.

Determining the actual weather is not so straightforward. This was done primarily by looking outside but with regard to rainfall and how heavy it was (drizzle, light rain, heavy rain etc) this was done by looking at the data coming out from a local weather station (about 2 miles away), Thatcham Weather Station. I'll make something clear here. Its the weather forecasters who say they can predict between light and heavy rain, if they'd just said "wet" or "dry" that would have made my life easier. However, there are scientific definitions for drizzle, light rain and heavy rain and these are what I used based upon the weather station data.

Also, I took the worst weather during the day, so if it was sunny in the morning and raining in the afternoon then I would assess the weather for that day as "rain". This is what the forecasters do anyway. I took the weather to be a 12 hour period from 6 am to 6 pm, however, I did give them a little leeway. On one day it started raining at 5:30 pm, so I assessed the forecast of "cloudy" to be correct in that instance. Also, with regard to white cloud or grey cloud, I didn't differentiate between the two. Similarly, if they said "sunny intervals" I gave them the benefit of the doubt if the sun only briefly showed itself.

Results

The image shows the forecast and actual weather for the period. Note that at the start I didn't have a full 5 day forecast, I had to build that up over the 5 days. The columns show the forecast for each day at 1 day, 2 days ... 5 days in advance. Ideally this should stay the same. The fact that it changes means that the weather forecasters are changing their minds.

The green background shows where the forecast was accurate, the red where it was wrong. The actual weather is in the top row. (Click on the picture for a bigger image).

The following image shows a summary of right and wrong.

So overall, the accuracy of the forecasts is:

	Days in advance	Accuracy (%)
	1 (actually 0-12 hours)	57
	2	23
	3	0
	4	0
	5	20

It could be argued that in order to get the forecast correct 5 days in advance, you also have to get it correct 2, 3, 4 days in advance as well. If you subscribe to this argument then the accuracy diminishes somewhat.

	Days in advance	Accuracy (%)
	1 (actually 0-12 hours)	57
	2	8
	3	0
	4	0
	5	0

With regard to Metcheck, their servers went down on day 2 so I only got 1 day's worth of data, here it is: As can be seen, based on a statistical sample size of 1, its no better or worse than the BBC.

Summary

To be perfectly blunt, 5 day forecasts are just rubbish. To be fair, if the 2 week period was in the summer and a big fat high pressure was sat over the UK then the 5 day forecast would undoubtably be better. What was surprising was the accuracy (or lack there of) of the forecast for the next 12 hours. On one day I was noting the forecast down and just laughed, you think "why don't they just look out of the window?"

Home page