Mean Absolute Percentage Error for regression models

Abstract: We study in this paper the consequences of using the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. We prove the existence of an optimal MAPE model and we show the universal consistency of Empirical Risk Minimization based on the MAPE. We also show that finding the best model under the MAPE is equivalent to doing weighted Mean Absolute Error (MAE) regression, and we apply this weighting strategy to kernel regression. The behavior of the MAPE kernel regression is illustrated on simulated data.

• The paper proves that an optimal MAPE model exists, establishing MAPE as a robust error measure analogous to MSE regression.
• It demonstrates that optimizing MAPE is equivalent to weighted MAE regression, with weights inversely related to the target magnitude.
• The study confirms the universal consistency of ERM under MAPE and validates its practical use through kernel quantile regression methods.

Insights from "Mean Absolute Percentage Error for Regression Models"

The paper "Mean Absolute Percentage Error for Regression Models" authored by Arnaud de Myttenaere, Boris Golden, Bénédicte Le Grand, and Fabrice Rossi, presents a comprehensive study of the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. The study explores both the theoretical and practical facets of using the MAPE, offering a robust analysis of its implications in machine learning.

Theoretical Contributions

The paper establishes several significant results regarding the MAPE in the context of regression models:

1. Optimal MAPE Model Existence: The authors prove the existence of an optimal model for MAPE, akin to the regression function for MSE. This is vital as it reassures that an optimal, well-defined model can be obtained within this error measure framework.
2. Equivalence to Weighted MAE: A critical equivalence is drawn between finding the best model under MAPE and performing weighted Mean Absolute Error (MAE) regression. Specifically, the MAPE can be transformed into a weighted version of the MAE where weights are inversely related to the target variable's magnitude.
3. Universal Consistency of ERM: The universal consistency of Empirical Risk Minimization (ERM) using MAPE is proven, showing that the estimated model approaches the optimal risk as the sample size increases. This result is crucial because it sets theoretical grounds for reliably using MAPE in practical predictive modeling environments.
4. Complexity Aspects and Capacity Measures: The paper investigates the effects of using MAPE on model complexity measures such as covering numbers and Vapnik-Chervonenkis (VC) dimensions. It highlights that MAPE-based measures retain upper bounds similar to those of the MAE, indicating that the transition to MAPE does not significantly affect the complexity of the utilized function classes.

Practical Implications and Kernel Regression

Moving from theory to practice, the authors illustrate how MAPE regression can be realized concretely using kernel methods. They develop an approach to adapt kernel quantile regression for MAPE:

1. Kernel Quantile Regression to MAPE: The paper shows that MAPE regression can be tackled using similar optimization strategies to those used in quantile regression but with appropriate instance weights. This allows for leveraging existing computational tools and frameworks—like the quantreg R package—, ensuring that the step from theory to application is both feasible and efficient.
2. Simulation Analysis: The behavior of MAPE kernel regression models is empirically examined through simulated data experiments. The results support the theoretical findings, evidencing the improved performance of MAPE-based regression in contexts where the target variable is strictly positive and relatively stable, which validates its appropriateness for applications like price prediction.

Implications and Future Research

This research marks a significant step toward embracing MAPE in regression tasks. Practically, it offers a viable alternative to MSE and MAE, particularly in domains sensitive to relative error interpretations, such as finance and demand forecasting. Theoretical assurances about the existence of optimal MAPE models and the consistency of MAPE-based ERM solidify the approach's credibility.

Future research might explore lifting the lower bound on the target variable, which is currently a prerequisite for theoretical results within the MAPE framework. Additionally, extending these findings to regularized risk minimization procedures could further align theoretical advancements with practical scenarios where regularization is crucial to manage overfitting.

Conclusion

The paper provides a rigorous and multifaceted examination of MAPE for regression models, binding theoretical robustness with practical applicability. By proving key theoretical properties and demonstrating efficient implementation methods, it opens the door for broader use of MAPE in regression analytics, fostering more accurate performance evaluation in contexts necessitating relative error sensitivity.