Fixing the Double Penalty in Data-Driven Weather Forecasting Through a Modified Spherical Harmonic Loss Function
Description
🎙️ Abstract:
Recent progress in data-driven weather forecasting has surpassed traditional physics-based systems. Yet, the common use of mean squared error (MSE) loss functions introduces a “double penalty,” smoothing out fine-scale structures. This episode discusses a simple, parameter-free fix to this issue by modifying the loss to disentangle decorrelation errors from spectral amplitude errors.
🌪️ Data-driven weather models like GraphCast often produce overly smooth outputs due to MSE loss, limiting resolution and underestimating extremes.
⚙️ The proposed Adjusted Mean Squared Error (AMSE) loss function addresses this by separating decorrelation and amplitude errors, improving spectrum fidelity.
📈 Fine-tuning GraphCast with AMSE boosts resolution dramatically (from 1,250km to 160km), enhances ensemble spread, and sharpens forecasts of cyclones and surface winds.
🔬 This shows deterministic forecasts can remain sharp and realistic without explicitly modeling ensemble uncertainty.
Redefining the loss function in data-driven weather forecasting can drastically sharpen predictions and enhance realism—without adding complexity or parameters.
📚 Citation:
https://doi.org/10.48550/arXiv.2501.19374
🔍 Bullet points summary:💡 Big idea:
United States
