Differentiating Through Linear Solvers
Abstract: Computer programs containing calls to linear solvers are a known challenge for automatic differentiation. Previous publications advise against differentiating through the low-level solver implementation, and instead advocate for high-level approaches that express the derivative in terms of a modified linear system that can be solved with a separate solver call. Despite this ubiquitous advice, we are not aware of prior work comparing the accuracy of both approaches. With this article we thus empirically study a simple question: What happens if we ignore common wisdom, and differentiate through linear solvers?
- An updated set of basic linear algebra subprograms (blas). ACM Transactions on Mathematical Software, 28(2):135–151, 2002.
- Christianson, B. Reverse accumulation and attractive fixed points. Optimization Methods and Software, 3(4):311–326, 1994.
- Christianson, B. Differentiating through conjugate gradient. Optimization Methods and Software, 33(4-6):988–994, 2018.
- The university of florida sparse matrix collection. ACM Trans. Math. Softw., 38(1), dec 2011. ISSN 0098-3500. doi: 10.1145/2049662.2049663. URL https://doi.org/10.1145/2049662.2049663.
- Differentiating the method of conjugate gradients. SIAM Journal on Matrix Analysis and Applications, 35(1):110–126, 2014.
- Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, 2008.
- Derivative convergence for iterative equation solvers. Optimization methods and software, 2(3-4):321–355, 1993.
- The tapenade automatic differentiation tool: Principles, model, and specification. ACM Transactions on Mathematical Software (TOMS), 39(3):1–43, 2013.
- Saad, Y. SPARSKIT: a basic tool kit for sparse matrix computations, 1994.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.