The trouble with weather apps

Software
1

…is data.

There have been many threads here about people’s favourite weather apps, and I’ve seen some I really like the look of, but a recent experiment I did shows the problem I have with all of them.

I was re-arranging my iPhone home screens; evicting some apps I only had present for recent travel, and some that I’m just not using enough to warrant a home screen spot (most apps live only in my App Library).

Anyway, I ended up with some extra space on a home screen and decided to revisit what widgets might prove useful. I discovered that I had a beautiful weather app called Overlook still installed, so I placed it on my home screen — alongside Apple’s 1st party app, and my national government weather provider’s app.

On Sunday morning I was greeted with this.

IMG_4891
IMG_48911125×1214 59.8 KB

I’ve scrubbed out the location in the MetService app because it can only display my current location and it (pointlessly) displays my actual suburb rather than the more generic “Wellington”. I have checked many times that the weather in “my suburb” and the weather in “Wellington” are always identical.

So we can see they all agreed the current temperature was 15º. Excellent.

But, the low temperature is 11º, 12º, or 13º depending on who you believe. The high is 17º, 18º, or 19º. The predominant conditions are either cloudy or windy or both.

Only Overlook is showing the rain chance, but they did all agree on 0%. They do not agree, however, when the percentage is above zero.

This morning (Monday) I took another look and the current temperature was different between all three, where all three agreed with at least one other on the high or low. So they’re not even consistently inconsistent with one another.

I could argue that Overlook shows only cloud and wind in the screenshot and that would imply overcast, but I’m not familiar with their symbology, so will spot it that point.

This is all over and above any app disagreeing with itself over time. Rain tomorrow? Don’t make that judgement until tomorrow, and even then it could be wrong. (Just last week, the next day’s rain was showers in the afternoon, then in the morning it was drizzle in the morning, and the actuality was drizzle then rain most of the day.)

I think all reviews of weather apps should focus on features and the understandability of the interface, with maybe just a footnote from the reviewer that says “The weather was accurate (or not) at my place.”

There is one app, however, that I have added below these, and whose data I put a bit more faith in…

IMG_4895
IMG_48951125×567 67.6 KB

This particular widget from AeroWeather is called a “Meteogram”. I looked it up and it’s a general term for this style of display of weather information — multiple factors in layers, time in the other axis.

The data source is my local airport. I know that it is in an exposed location, so I mentally have to adjust for that, but aviation weather forecasters need to be pretty darned good at their predictions. My only problem is the cloud row seems to always just display :m: which I have not figured out.

This screenshot was taken just now and seems to show a low of 11º tonight. Of course, the other apps disagree on this. Overlook says 12º and the other two say 13º. Of course.

1 Like
2

You know, predictions are always tricky, especially when they are about the future :slight_smile:

As you said, there are multiple data sources used, multiple prediction models that have varying degrees of resolution. A forecast is an indication of likely outcomes - not a promise of exact outcomes.

Maybe stick with a single weather app?

5 Likes
3

This turned out to be a very long version of @airwhale 's comment.
There is no such thing as an accurate weather forecast. The trouble with weather apps is that they are totally dependent on the quality of the data they buy from weather forecasters, and those forecasters do NOT produce identical forecasts for a number of reasons:

  • some are better than others and people producing forecasts for critical purposes (e.g. aviation) tend to be better but also tend to charge more for their data, with some charging much more when the data is more granular (i.e. locally accurate). Some apps are using data points 100km apart meaning that it may be showing the weather an average of 50km from you. Similarly, some are updating current data every hour and many are updating it every 12 hours and interpolating.
  • weather is a chaotic system. The tiniest difference in input can result in large difference in output. Forecasts work on probabilities which indicate uncertainty: if a forecast model is showing a rain probability of 5% that simply says that rain isn’t impossible but unlikely: but if it rains people are annoyed and feel the whole forecast is wrong. Some weather is generated very locally (e.g. thunderstorms and showers) and they really can’t do more than say that these are likely to occur over a broad area but there is no way to tell exactly where until storms form.
  • weather forecasts are generated from very complex mathematical models which are fed with real-world data (modelling the future atmosphere by applying known physics to the existing atmosphere and then rolling that forward repeatedly). Because of the chaotic nature of weather a model will produce slightly different results in different runs from the same data and often very different results from runs with very slightly different data, especially as the future gets further (in hours or days) from the present data. Forecasters use “ensemble” forecasting, where they run the models many times, get all the varied forecasts and then look at the “average” of them. The more runs, the more likely to be accurate. That’s why these people need supercomputers. The trouble is that sometimes the weather is actually closer to one of the outliers.

It is impossible to predict weather accurately for lots of places. It is possible to be accurate in predicting weather for one place (e.g. an airport), maybe a few hours ahead. Different apps will use different data sources, at different levels of granularity and different rates of update.

My approach is to use one app consistently (I use Apple weather) and learn over time how to interpret it. I know for my town it is often wrong when it predicts light or very light rain, but can be accurate to the minute with heavier rain, for example, and is generally within a degree for temperatures.

7 Likes
4

Totally agree with what others have said.

Having a low range of 2º between sources is surely within a tolerance that would allow users to make the same informed decisions.

4 Likes
5

For the most part my use of weather apps is to check if/when/how-long it is going to rain here. Primarily to dress appropriately for my daily walk/exercise; will a tee-shirt surice or must I don a sweater or wear my waterproof jacket.

I use four apps and (mentally) aggregate the results. These iOS apps are: Apple Weather, OpenWeather, MetOffice — I am in the UK, and BBC Weather. All of them use GPS on my iPhone to give me predictions of local conditions. Typically I rely on OpenWeather as it gives me a prediction of whether there will be precipitation in the next hour. Supplement this with Apple Weather and MetOffice as they both include animated predictions of where rain is going to fall. (If you follow Formula 1 like them I rely on animated rain displays.) Since the BBC stopped using Met Office data for its forecasts I have given their predictions less weight because they can be very wide of the mark here.

Recently we had a major storm come through the south of England. The Met Office issues warnings about the effects of the storm. My location was on the cusp of the storm so one day a be-aware warning was in effect, the next there was no warning, then it there was none. All this as the meterologists were modelling the track of the storm.

At best a weather forecast is nothing more than a statistical prediction and each agency uses a different model. At worst the forecast is total fiction.

6

You may find this article helpful.

And this:

7

Just for interest. This was that storm on the South Coast of England, a few miles from my house.

We were under a “direct danger to life” warning which was later changed to an amber warning (disruption and potential danger) as the storm went slightly further south than expected. Weather is chaotic: it changes itself as it happens.

2 Likes
8

@chrisecurtis @zkarj SInce I posted my earlier reply the Met Office issued an Amber warning for thunderstorms. This has now been changed to the lesser Yellow warning as the storm tracks through. By tomorrow morning it might have been removed as again my location is right on the cusp of the affected area.

1 Like
9

+1

While I was walking this morning I made a very unscientific comparison.

  • According to aviationweather.gov it was 36°F at our local airport

  • The Windy app reported the same 36° at the airport,
    and also 39° from a location closer to me.

  • The Google Assistant said it was 40°

  • Apple Weather said it was 44°

  • And Siri said it was 70°

10

Wow. Surely they use the same source!?

1 Like
11

I have no idea. A couple of years ago I asked the temperature, was told it was 51°, and that “there is currently a typhoon warning in effect”. We don’t get a lot of those in south eastern US. :grinning:

2 Likes
12

Maybe Siri thought you asked for the indoor temperature.

13

Maybe. I was outbound, at least a half mile from home, with the Outdoor Walk workout running on my Apple Watch. I guess “she” could have forgotten to check GPS. :wink:

Seriously, computers sometimes glitch and we never know why. When I’m out exercising Siri normally gives me a temperature report that sounds reasonable.

1 Like
14

I’ve had the storm today (in England) and my weather app didn’t give me a notification of my weather warning at all, so that was a bit of a weather fail :roll_eyes:

I think we need to make a distinction between the weather app, and the weather app’s data source. Over the last 12 months I’ve realised it’s all about the latter, and that’s not the shiny bit that makes people buy weather apps.

A good weather app should tell you what data source it uses, and a great weather app should give you options and let you pick your preferred source. For our fellow Kiwis and Aussies, this must be essential, because you need a data source that is retrieving local data.

I switched to the Foreca data source this year and have been reasonably happy with it. I wondered how they got their forecast so close to accurate for my location, and it turns out the answer is that Foreca’s UK data is coming from the Met Office, which is our national weather service. Foreca is produced by a Finnish company and favours European forecasting, though it does get U.S. data and some Pacific data as well (not enough to be of use for you @zkarj unfortunately, they only include Taiwan and South Korea).

I am currently using Weathergraph as my app, and aside from not giving me wind warnings it is a pleasure to use. For me, even if I change weather app (I’m not feeling inclined to at present), I would stick with the Foreca dataset, so I will only move to another app which provides that as a source.

3 Likes
15

Another thought is to use as many as possible and look for a consensus, which I see has been suggested by others.

Which was really the point of my (laboured) post… weather app reviewers need to keep that thought front of mind when writing about the apps.

This is, of course, 2ºC, which is 3.6ºF. I agree that it’s difficult to tell the temperatures I have shown apart, in actually feeling them outside, however if it’s the difference between 2º and 4º then it’s actually quite important as a predictor of ice. Also, the difference between 17º and 19º is noticeable to me.

Me, too. It’s hard enough that my home and office get quite different weather, despite the single forecast area, but as I mentioned, the rain predictions, even in the broad strokes like “showers in the morning” are woefully inaccurate here because all the services are predicting across the entire country. Our local university is, by reputation, scarily accurate with their rain predictions, but I’ve not found a way to use them as a data source and their own presentations are fit for meteorologists and not mere mortals like me.

Thanks, I will take a read, although to my main point (I called out above), “most accurate” articles, in my experience, are true for relatively specific areas, usually the continental USA. I do wish there were reputable Apple or even general tech publications that had journalists all around the world that could ensure these kinds of biases (and I don’t say that in a nasty way) could be eliminated.

On that… who knew, for instance, that the new “Siri” as distinct from “Hey Siri” is available in only USA, UK, Canada, and Australia? Not me, based on all I have read.

1 Like
16

Every time I open Mac Power Users and see this thread I think “The trouble with weather apps is they aren’t input devices”. I want to open the app, tap on “Sunny” and “70ºF”, go outside and enjoy the weather I selected. What good is a weather app that will only tell you what the weather is?

9 Likes
17

All my weather apps are just as accurate as the local tv weather people. In fact, the locals like to exaggerate when rain is a possibility so people tune back in to get updates. When it’s going to be sunny for a number of days, they looked bummed. No one watches the weather when they know it’s going to be nice for 5 or more days.

18

The latest storm (Debi) passed me by, other than a morning of heavy rain. The Yellow warning for thunderstorms was changed during the morning from a finish of 12noon to finish at 3PM, which happened to be when the rain stopped. By 3:30 it was a nice afternoon and I managed my daily walk of nearly three miles at a pace slightly better than 3MPH.

Of the four apps I use two are based on national metreological organisations. MetOffice on the UK’s Metreological Office obviously. BBC Weather on their French counterpart Meteo; though in the past they used the MetOffice. I believe that these two share their data as part of EuroMet. OpenWeather explain their sources here Accuracy and quality of weather data - OpenWeatherMap From the little I know about weather forecasting they use a machine learning apporach where as national/international metreological services use strictly mathematical models. Apple list their data sources here Feature availability and data sources in the Weather app – Apple Support (UK) For the UK all of them pick up the Met Office’s adverse weather warnings.

OpenWeather appears to base their forecasts on raw data. They are also the only one that offers access to historical data.

I have been forgetting to mention one other source that I use — looking out of the window. Most weather systems flow from west to east over my house. Easy to confirm any of the apps predictions with the facts. And a quick “back of the envelope” — simply observing how fast the clouds are moving — corroberates or contradicts the forecast of whether there will be rain in the next hour (which is the duration of my exercise).

19

Apparently, artificial intelligence will help improve current weather forecasting techniques.

1 Like
20

What good is a weather app that will only tell you what the weather is?

Perhaps that’ll be an add-on once the Apple Vision Pro comes out. ;D