The Self-Simplifying Machine: Exploiting the Structure of Piecewise Linear Neural Networks to Create Interpretable Models

Abstract: Today, it is more important than ever before for users to have trust in the models they use. As Machine Learning models fall under increased regulatory scrutiny and begin to see more applications in high-stakes situations, it becomes critical to explain our models. Piecewise Linear Neural Networks (PLNN) with the ReLU activation function have quickly become extremely popular models due to many appealing properties; however, they still present many challenges in the areas of robustness and interpretation. To this end, we introduce novel methodology toward simplification and increased interpretability of Piecewise Linear Neural Networks for classification tasks. Our methods include the use of a trained, deep network to produce a well-performing, single-hidden-layer network without further stochastic training, in addition to an algorithm to reduce flat networks to a smaller, more interpretable size with minimal loss in performance. On these methods, we conduct preliminary studies of model performance, as well as a case study on Wells Fargo's Home Lending dataset, together with visual model interpretation.