The humble matrix multiplication along with its inverse is almost exclusively what’s going on in many simple ML models
This is the fourth chapter of the in-progress book on linear algebra, “A birds eye view of linear algebra”. The table of contents so far:
- Chapter-1: The basics
- Chapter-2: The measure of a map — determinants
- Chapter-3: Why is matrix multiplication the way it is?
- Chapter-4 (current): Systems of equations, linear regression and neural networks
All images in this blog, unless otherwise stated, are by the author.
Modern AI models leverage high dimensional vector spaces to encode information. And the tool we have for reasoning about high dimensional spaces and mappings between them is linear algebra.
And within that field, matrix multiplication (along with its inverse) is literally all you need to build many simple machine learning models end to end. Which is why spending the time to understand it really well is a great investment. And this is what we did in chapter 3.
These simple models, useful in their own right, form the building blocks of more complex ML and AI models with state of the art performance.
We’ll cover a few of these applications (from linear regression to elementary neural networks) in this chapter.
But first, we need to go to the simplest case in the simplest model — when the number of data points equals the number of model parameters. The case of solving a system of linear equations.
We have finally arrived (in the context of this book) at the heart of linear algebra. Solving systems of linear equations is how we discovered linear algebra in the first place and the motivations for most concepts in this field have deep roots in this application.
Let’s start simple and one dimensional. The concept of division is rooted in one…