NOTE 0015 / RESEARCH
How many days ahead is a weather forecast worth trusting?
Every weather app shows you a number and never shows you whether that number was worth printing. So I scored 7,358,400 archived forecasts against a competitor that needs no model at all, across thirty Turkish cities and two years of hourly data.
The competitor is deliberately stupid. For rain it is the sentence "it will not rain" — which is true most hours, because rain is rare. For temperature it is "tomorrow will be like today." Both are free. The question is the lead time at which the real forecast stops beating them. Call it the trust horizon.
In eight of the thirty cities, the hourly rain forecast never beats "it will not rain" — not at one day ahead, and not at seven.
The table
Trust horizon in days, with a bootstrap interval from 4,000 resamples. Zero means the forecast did not win at any lead the archive carries.
city rain rain CI temp temp CI ------------------------------------------------------------ Adana 3.36 [1.45, 4.06] 4.97 [4.69, 5.4] Ankara 0.00 [0.0, 1.0] 5.92 [5.58, 6.26] Antalya 2.28 [0.0, 4.18] 0.00 [0.0, 0.0] Aydin 2.71 [1.84, 3.42] 4.16 [4.04, 4.28] Balikesir 2.03 [1.3, 2.7] 5.50 [5.08, 5.94] Bursa 3.43 [2.89, 3.99] 5.98 [5.62, 6.55] Canakkale 2.22 [1.66, 2.89] 5.68 [5.08, 7.0] Denizli 0.00 [0.0, 1.29] 4.27 [4.11, 4.44] Diyarbakir 1.72 [0.0, 2.84] 5.02 [4.51, 5.52] Erzurum 1.39 [0.0, 2.25] 3.97 [3.32, 5.43] Eskisehir 0.00 [0.0, 1.32] 6.25 [5.88, 6.64] Gaziantep 2.64 [1.44, 3.71] 6.17 [5.8, 6.51] Isparta 0.00 [0.0, 0.0] 4.23 [4.11, 4.34] Istanbul 2.82 [1.78, 3.59] 5.51 [4.99, 6.13] Izmir 2.86 [1.72, 4.11] 5.73 [5.39, 6.5] Kars 0.00 [0.0, 0.0] 4.35 [3.74, 4.91] Kayseri 0.00 [0.0, 1.21] 4.97 [4.62, 5.35] Konya 0.00 [0.0, 1.37] 6.13 [5.88, 6.31] Malatya 2.71 [1.8, 4.09] 4.80 [4.56, 5.09] Mersin 1.75 [0.0, 3.71] 0.00 [0.0, 2.33] Mugla 5.61 [4.55, 6.55] 4.50 [4.35, 4.66] Ordu 4.59 [4.14, 5.06] 4.90 [4.74, 5.19] Rize 5.74 [5.11, 6.71] 4.17 [4.06, 4.28] Samsun 3.18 [1.96, 3.57] 4.72 [4.51, 4.96] Sanliurfa 1.18 [0.0, 2.47] 5.99 [5.55, 6.21] Sivas 1.81 [0.0, 2.91] 5.55 [5.15, 5.97] Tokat 2.63 [1.52, 3.4] 5.77 [5.34, 6.31] Trabzon 4.45 [3.95, 4.88] 7.00 [5.75, 7.0] Van 0.00 [0.0, 1.41] 4.15 [3.88, 4.32] Zonguldak 4.40 [3.9, 4.9] 4.84 [4.69, 5.01]
The eight cities where rain forecasting loses
Ankara, Konya, Kayseri, Eskişehir, Denizli, Isparta, Kars and Van. All inland, all dry. In every one of them measured skill is negative at all seven leads.
Two of those are stronger claims than the other six. In Isparta and Kars the bootstrap interval is [0.00, 0.00] — no resample produced a positive horizon. In the remaining six the interval reaches about a day, so the honest sentence is "measured negative at every lead", not "provably never". I am keeping the distinction because it is the whole difference between a result and a headline.
Why a dry city scores zero
This is not a story about bad models. It rains in Ankara roughly one hour in twenty, so "it will not rain" is already right about 95% of the time before anyone looks at anything. To clear that bar a forecast has to catch enough real rain to pay for every hour it cries wolf, and at hourly resolution that is a hard trade.
In Rize, where it rains far more often, the naive rule is weak — and the forecast is worth reading almost six days ahead. Same models, same archive, opposite answer.
The result is mostly not about forecast quality. It is about how good the free guess already is where you live.
Ankara makes the point twice in one row. Its temperature horizon is 5.92 days and its rain horizon is zero. Same city, same two years, same providers — the answer depends entirely on which question you ask, and on what the lazy alternative happens to be for that question.
The two cities where temperature loses
Antalya and Mersin, and the mechanism is the mirror image. On that coast tomorrow really is like today: the seasonal drift is slow and the daily range is narrow, so persistence is an extremely strong baseline. Antalya's interval is [0.00, 0.00]; Mersin's reaches 2.33 days, which makes Antalya the firmer of the two.
Meanwhile Trabzon's temperature forecast is still winning at 7.0 days — the last lead the archive carries, so the true horizon there is somewhere beyond what I can measure. The table's upper bound is a property of the dataset, not of the forecast.
How it was built
Python fetch and shape network-bound, nothing else matters Rust skill and bootstrap 30 cities x 2 variables x 7 leads x 4,000 resamples Python analysis and charts
The split is not ideological. Fetching two years of archived forecasts for thirty cities is entirely waiting on a network, and Python is the shortest path to that. The scoring is 1.68 million bootstrap evaluations, which is the part where the language choice shows up in wall-clock time rather than in taste.
Every horizon in the table carries an interval because a single skill number, at a single lead, on two years of one city, is not a measurement anybody should act on. The intervals are what let me say Isparta and Ankara are different kinds of zero.
What it is good for
Concretely: if you live in one of those eight cities, the hourly rain probability in your weather app carried no information over "probably not" for the two years I measured. Temperature is a different story almost everywhere — five to six days is typical, and that is genuinely useful.
More generally, this is the shape of question I think gets skipped most often. Everyone measures whether a model is accurate. Far fewer measure whether it beats the thing you would have done without it, and that second number is the one that decides whether the model should exist.
It also has to be measured per segment, not in aggregate. A single national average across these thirty cities would report a positive rain horizon and hide the eight cities where the honest answer is zero.