Linear algebra is very fundamental and critical in machine learning and data mining research. Essence of Linear Algebra gives a very good description of the basic concepts. Here in this article, I added some comments and thoughts. This is mainly for helping myself understand and memorise the details better.

I. Linear Transformation of a Coordinate System:

What I would like to add concerning linear transformation is another explanation of how the linear transformation matrix comes. Typically, for a coordinate system, the basis vectors are known, e.g., $\left ( x_1, x_2, \cdots, x_n \right )$ , where each $x$ is a basis vector and $n$ is the number of basis vectors and also the number of dimensions usually. This basis vector set defines the coordinate system and any vector in this coordinate system can be expressed using:

$(x_1, x_2, \cdots, x_n) \begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix}$ ,

in which the column vector with all $\xi$ is the coordinate for the given vector.

A linear transformation of a coordinate system can also be considered as the linear transformation of the basis vectors in the coordinate system, which transforms the original set of basis vectors, e.g., $\left ( x_1, x_2, \cdots, x_n \right )$ , into another set, e.g., $(y_1, y_2, \cdots, y_n)$ , defining a new coordinate system. And the relationship between these two sets could be formalised as:

$(y_1, y_2, \cdots, y_n) = (x_1, x_2, \cdots, x_n)\begin{bmatrix} a_{11} & a_{21} & \cdots & a_{n1} \\ a_{12} & a_{22} & \cdots & a_{n2} \\ \cdots & \cdots & \cdots & \cdots \\ a_{1n} & a_{2n} & \cdots & a_{nn} \end{bmatrix}$ ,

where one can see that a new basis vector is a linear combination of the original basis vectors. With the linear transformation of the whole space, the coordinate for a vector in the original space will also be transformed such that:

$(x_1, x_2, \cdots, x_n)\begin{bmatrix} a_{11} & a_{21} & \cdots & a_{n1} \\ a_{12} & a_{22} & \cdots & a_{n2} \\ \cdots & \cdots & \cdots & \cdots \\ a_{1n} & a_{2n} & \cdots & a_{nn} \end{bmatrix} \begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix} = (x_1, x_2, \cdots, x_n)\begin{bmatrix} \eta_1\\ \eta_2\\ \cdots\\ \eta_n \end{bmatrix}$ ,

showing that the new coordinate $\xi'$ (from the perspective of the original space) of a vector is equal to the linear transformation matrix $A$ (the matrix with all $a$ s) times the original coordinate $\xi$ : (in matrix form)

$\xi' = A \xi$ .

Note that this explanation does not apply well when the linear transformation matrix is non-square. Therefore, the intuition provided by the video shown above may be a better way to memorise linear transformation, in which:

each column vector in the linear transformation matrix is the target vector into which the corresponding original basis vector are transforming. More formally, this could be interprete as directly changing the basis vectors into another set of basis vectors of the same number:
$(x_1, x_2, \cdots, x_n) \Rightarrow (x'_1, x'_2, \cdots, x'_n)$ ,
such that we have:
$(x_1, x_2, \cdots, x_n) \begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix} \Rightarrow (x'_1, x'_2, \cdots, x'_n) \begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix}$ .
The new basis vectors could be of a different dimension and the coordinate of the original vector after transformation becomes:
$\begin{bmatrix} \xi'_1\\ \xi'_2\\ \cdots\\ \xi'_n \end{bmatrix} = (x'_1, x'_2, \cdots, x'_n) \begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix}$ ,
i.e., the direct multiplication of the transformed basis vectors and the original coordinates.
the new coordinate of a vector (from the perspective of the original space) after transformation is simply the transformation matrix times the original coordinate.

II. Linear Transformation of two Coordinate Systems:

In last part, all the coordinates we used, e.g., $\xi, \xi'$ , are from the perspective of the original space, i.e., standard basis vectors that form the standard coordinate system. Note that to communicate with a different coordinate system, the coordinates should be transformed such that two coordinate systems are using different coordinates to describe a same vector. This is explained in the following video:

Basically, there are three operations that are worth further emphases. Note that we are using $\eta, \eta'$ to denote the coordinates in a new coordinate system other than the standard coordinate system.

from coordinate $\eta$ of a new coordinate system to that $\xi$ of the standard coordinate system:
$\begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix} = \begin{bmatrix} \mathbf{b_1} & \mathbf{b_2} & \cdots & \mathbf{b_n} \end{bmatrix} \begin{bmatrix} \eta_1\\ \eta_2\\ \cdots\\ \eta_n \end{bmatrix}$ ,
where $\eta$ is the coordinate of the new coordinate system defined by $\mathbf{b}$ s, $\mathbf{b}$ s are the coordinates of the basis vectors of the new coordinate system from the perspective of the standard coordinate system, and $\xi$ is the coordinate of the standard coordinate system. This whole process has the same formulation as that of linear transformation, but maintains a very different geometric meaning. In linear transformation, this operation stands for the change of vector, e.g., rotation, scaling, etc. However, in change of coordinate system, this operation stands for the translation of a same vector from one system to another, i.e., the vector itself is intrinsically unchanged;
from coordinate $\xi$ of the standard coordinate system to that $\eta$ of the new coordinate system:
$\begin{bmatrix} \mathbf{b_1} & \mathbf{b_2} & \dots & \mathbf{b_n} \end{bmatrix}^{-1} \begin{bmatrix} \xi_1\\ \xi_2\\ \cdots\\ \xi_n \end{bmatrix} = \begin{bmatrix} \eta_1\\ \eta_2\\ \cdots\\ \eta_n \end{bmatrix}$ .
This process is straightforwardly derived from last process with the assumption that the inverse exists. Therefore, one has to make sure that the two coordinate systems are of the same dimension and convertible to each other through linear transformation.
from coordinate of the new coordinate system to that of the standard coordinate system, linear transform and then convert back to that of the new coordinate system, i.e., performing linear transformation in the new coordinate system:
$\eta' = \mathbf{B}^{-1}\mathbf{A}\mathbf{B}\eta =\\ \begin{bmatrix} \mathbf{b_1} & \mathbf{b_2} & \cdots & \mathbf{b_n} \end{bmatrix}^{-1} \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n}\\ \cdots & \cdots & \cdots & \cdots\\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{bmatrix} \begin{bmatrix} \mathbf{b_1} & \mathbf{b_2} & \cdots & \mathbf{b_n} \end{bmatrix} \begin{bmatrix} \eta_1\\ \eta_2\\ \cdots\\ \eta_n \end{bmatrix}$ ,
where the $a$ s form the linear transformation matrix from the perspective of the standard coordinate system, and $\eta'$ is the target coordinate in the new coordinate system after linear transformation. A very helpful use case of the above transformation is provided in the above video.