ReLU activated neural networks have a lot in common with non-smooth dynamical systems! Building off our prior work on frictional robotic systems, we analyze the stability of learned neural network control policies using convex optimization, specifically Linear Matrix Inequalities (LMIs). This efficient approach is made possible by drawing a clear connection between these neural networks and Linear Complementarity Systems. Feedback is welcome! The paper is below, with code to come shortly.