MAE, RMSE and WAPE are often compared as though one of them must be the “correct” forecasting metric. The reality is less convenient. Each metric answers a different question about error, and none of them knows by itself how much that error costs the business. The choice matters because the same forecasts can look reasonable under one metric and weak under another. A model that avoids large misses may be favoured by RMSE; one with stable absolute deviations may look better under MAE; WAPE can provide an intuitive aggregate view. The metric does more than score the model: it defines which kind of failure receives attention.
What each metric measures
MAE averages absolute errors. Its main advantage is direct interpretation in the original units. If demand is measured in units, MAE gives a practical sense of how many units the forecast misses by on average. It is simple, robust and easy to communicate, but it treats increases in error linearly. RMSE squares errors before aggregation and then returns to the original scale. Large misses therefore receive much more weight. That can be useful when extreme errors are particularly damaging, but a small number of episodes can dominate the comparison.
WAPE divides total absolute error by total actual demand. Its percentage form makes aggregate comparisons easier. The trade-off is that high-volume observations dominate the result. A large group of fast-moving products can mask weak performance on small but critical items.
Scale changes the interpretation
MAE and RMSE are scale dependent. An error of 100 units can be catastrophic for an item selling 200 units and almost irrelevant for one selling 100,000. Directly comparing heterogeneous products therefore requires care. WAPE improves relative comparability, but introduces a different issue: volume determines weight. If decisions are made by SKU, warehouse or customer, a portfolio-level WAPE may be too aggregated to represent the operating decision. Intermittent demand creates further complications. Series containing many zeros can make some percentage metrics unstable or difficult to interpret. Evaluation design should match the demand pattern and the granularity at which the company acts.
Business cost creates asymmetry
None of these metrics automatically distinguishes over-forecasting from under-forecasting. The business often does. Underestimating a critical component may stop production, while modest overestimation may only increase holding cost. For perishables, the asymmetry can reverse. That means two models with the same MAE can produce very different economic outcomes. If one tends to miss low and the other tends to miss high, replenishment decisions may respond very differently. Horizon matters as well. A large error three months out may still be correctable; the same error two days before an order cutoff can be expensive. Statistical metrics should therefore be complemented by horizon-specific evaluation and, where appropriate, a decision cost function.
Choose from the decision
A practical selection starts with the operating question. MAE is often a strong baseline when the team wants a stable, interpretable measure in original units. RMSE is useful when large errors deserve additional weight. WAPE can be valuable when the goal is a relative aggregate view linked to total volume. But none should be the only measure when an important economic decision sits downstream. Add segmentation by criticality, horizon and error direction, then test what happens when the forecast feeds the real policy.
Good practice is not to search for a universal metric. Build a small hierarchy instead: one primary statistical metric, several diagnostic measures and a business-impact measure. The model wins when the decision improves, not when it tops an accuracy table. A useful additional check is to repeat the comparison across meaningful portfolio slices. A model can look stronger overall while losing on critical items, intermittent demand or products with long lead times. The primary metric should survive a segmented reading. If the conclusion changes after those cuts, the metric is not necessarily wrong; the business simply contains more than one error regime and should treat them explicitly rather than compressing everything into a single score.