Two weeks of running the forecast service - pretty accurate, actually

In three lines

• For two weeks a home PC has run the 05:00 issuance on its own. In all 14 issuances our temperature, humidity and wind errors were smaller than the KMA’s
• Rain was won big on rainy days, but on days with only a few rainy hours we sometimes caught none
• Re-checking the KMA numbers before posting a Short, I found a bug on our side that added 0.05 to the KMA’s wind, and fixed it

Every day at 05:30, the home PC runs itself

Made by HSL — the daily job Windows Task Scheduler starts at 05:30

• At 05:30 Task Scheduler runs one script; nobody touches it
• The slowest step is running Aurora on a Colab L4 to build the box around Korea
• The forecast reached the site between 05:46 and 06:09, so a run takes 16–39 minutes
• On 5 of the 11 mornings from Oct 17 to 27 the Colab connection dropped mid-run. A retry that reconnects up to three times meant not a single day was missed

[05:30:05] Aurora 박스 (2026102705)
[colab] Session READY.
RuntimeError: Connection was lost.
[colab_bg] exec 끊김 (1/3)
...
[06:09:07] 올리기
[service] 16개 올림 -> https://hslab.space/api/weather

• The log is in Korean: “Aurora box”, “exec dropped (1/3)”, “upload”, “16 files uploaded”
• Observations arrive twice. The last hours before issuance are filled from the KMA nowcast, then overwritten by official ASOS observations the next day

Two weeks: 14 out of 14

Made by HSL — temperature MAE per issuance, 97 stations, as of Oct 27

• Temperature, humidity and wind: our error was smaller in all 14 issuances, not one day lost
• Our temperature error was 24% below the KMA’s, close to the 22% of the test period, which suggests the test result was not luck
• Scores cover the 62,638 station-hours (97 stations × 48 hours) where both the KMA value and an observation exist
• The Oct 26 issuance is only half scored so far, and light rain before Oct 20 comes from adding the classifier afterwards (part 5)

Made by HSL — last-7-days table on the project page, updated Oct 27 06:09 KST

Rain still swings

• Rainy days were big wins; on Oct 23 our score was more than double the KMA’s
• Days with only a few rainy hours went the other way. On Oct 21 we missed all 12 rainy hours while the KMA caught 6
• Our model calls rain only when the chance of 1 mm or more passes 0.55, and a passing shower rarely pushes it that high
• Our rain F1 was 0 in 5 of the 14 issuances. All five had 12 rainy hours or fewer, and in three of them the KMA also scored 0

The better we do, the harder I check the other side

• On Oct 24, before posting the Short, I re-checked one question: was the KMA really scored at each station’s own location?
• The locations were right. Station coordinates map to 5 km grid cells with the conversion in the KMA’s guide, and values in the same cell matched
• Wind was not. The grid files truncate wind to one decimal, so we added 0.05; from Oct 13 the same 0.05 was also being added to point values that need no fix
• The KMA tends to forecast wind too strong, so an extra 0.05 made its error look that much bigger

• I refilled 11 stored forecasts from the originals and rebuilt the site numbers; the ranking did not change
• Still, a number skewed against the KMA sat on the page for more than ten days, which did not feel good
• The two-week table above uses the fixed numbers

If I did it again

• Test the code that handles the opponent’s data as hard as our own model; one correction worked against the KMA for 11 days
• Do not read rain scores one day at a time; only periods with hundreds of rainy hours mean much
• Building in retries from the start was the best call; the connection dropped on 5 of 11 mornings and no forecast was missed
• The next part is the US version against the US National Weather Service; today’s forecast is on the project page