### Archive

Posts Tagged ‘linear algebra’

## Notes on diagonalizable linear operators

Okay, I have a few things in my mind about this topic and I don’t want to lose them tomorrow or the day after. Therefore, I created this note. Hope it is useful to other people as well. Background of this note are as follows:

1. Suppose $A$ is an $n \times n$ matrix and it is diagonalizable. Therefore, by definition, there is an invertible matrix $P$ and a diagonal matrix $D$. However, I always forget the relationship between $A$ and $D$, whether it is $D=PAP^{-1}$ or $D=P^{-1}AP$. Generally, those two are no different, but we can choose $P$ such that the column vectors of matrix $P$ are the eigenvectors of matrix $A$, hence we have to know precisely which one it is, otherwise it gets wrong. Until now, when I forget about this, I always derive it and it takes some of my precious time (haha) but now after I have this, I can just open the Internet and look for this note.
2. Point 1 above talks about diagonalizability of a matrix. But generally, we have a linear operator acting on a general vector field instead. A linear operator is called diagonalizable if there is a basis B such that the matrix representation of this linear operator is a diagonal matrix. It seems for me, diagonalizability of a linear operator and that of a matrix are talking about two different things. One is talking about how to find a basis, while the other is talking about how to find an invertible matrix. However, these two concepts turn out to be the same.

We begin with the following theorem.

Theorem 1 (Coordinate-change matrix)
Let $E$ be a standard basis and $B$ be another basis for an $n$-dimensional linear space $V$. Then there is an invertible matrix $P$ such that for every $v \in V$

$P (v)_B=(v)_E$.                                                                                                          (1)

Matrix $P$ is called the transition or coordinate-change matrix from the basis $B$ to the basis $E$. Note that $(v)_B$ and $(v)_E$ are the coordinates of vector $v$ with respect to basis $B$ and $E$ respectively. In fact, the column vectors of matrix $P$ are the coordinates of the basis vectors of $B$ with respect to the standard bases $E$ (i.e. if $B=\{ v_1,v_2,\dots,v_n \}$ then $P=[(v_1)_E (v_2)_E \dots (v_n)_E]$ ).

We won’t prove the theorem as it can be found in any linear algebra textbook. However, using this theorem we can relate the diagonalizability of a linear operator and that of a matrix.Suppose $T$ is a diagonalizable linear operator on an $n$-dimensional linear space $V$. Take $x \in V$ then let $y$ be $y=T(x)$. Suppose $E$ is a standard basis in $V$ and $B$ is a basis in $V$ that consists of independent eigenvectors of $T$.

Then we have the following.

(i) $(y)_E = [T(x)]_E = [T]_E (x)_E = A (x)_E$, where $A$ is a representation matrix of $T$ with respect to standard basis $E$.
(ii) $(y)_B = [T(x)]_B = [T]_B (x)_B = D (x)_B$, where $D$ is a representation matrix of $T$ with respect to basis $B$. We also know that $D$ is a diagonal matrix.

We want to show that using the theorem 1, $D=PAP^{-1}$ or $D=P^{-1}AP$. From equation (1) we get:

$P (y)_B = (y)_E = A (x)_E = A P (x)_B$.

Hence, we have $(y)_B = P^{-1}AP (x)_B$. While from point (ii) we have $(y)_B = D (x)_B$. As a conclusion, we have,

$D = P^{-1} A P$.

We have derived that in fact, $D = P^{-1} A P$ is the right one. We also derived that the invertible matrix $P$ has such a close relationship with the basis $B$ as the column vectors of $P$ are the basis vectors of $B$, which are the eigenvectors of the linear operator $T$. This conclude my note.

Categories: Justincase, Teachings

## The solvability condition of A x = b when A is not a square matrix

In an elementary linear algebra course, we learned the solvability condition of the linear system

$A \mathbf{x} =\mathbf{ b}$

where $A$ is an $n \times n$ matrix and $\mathbf{x}, \mathbf{b}$ are $n \times 1$ vectors.

To mention a few, the followings are the solvability conditions (equivalent conditions) such that the linear system above has a solution:

1. determinant of $A$ is not zero
2. rank of $A$ is full
3. all eigenvalues of $A$ are non zero
4. the vector columns (also rows) are independent
5. $A$ has an inverse
6. et cetera

However, I would like to discuss the case when either (#) $A$ is not a square matrix or
(##) the rank of $A$ is not full (< $latex n$).

There is actually a condition such that the linear system $A \mathbf{x} = \mathbf{b}$ has a solution whenever we have either (#) or (##). The solvability condition is

$\mathbf {b} \in Col(A)$.

The thing is this condition is not applicable to most problems I have ever had. For instances, I bumped into the following problem.

Let $A, \mathbf{x}, \mathbf{b}$ be the following

$A=\left(\begin{array}{cc}a_{11} & a_{21} \\a_{12} & a_{22} \\b_{1} &b_{2} \\\end{array}\right)$$\mathbf{x} = (l_1, l_2)^T$ and $\mathbf{b}=(-a_{11},-a_{22},0)^T$. I want to seek a condition such that the linear system $A \mathbf{x}=\mathbf{b}$ is solvable for $l_1,l_2$. I want to find a condition for $a_{ij}$ and $b_i$ such that there is a solution of $l_1, l_2$.

The statement $\mathbf{b} \in Col(A)$ did not help me at that time and I look for another implication for this statement that is applicable enough to my problem.

Fundamental Theorem of Orthogonality (see [1])
Let $A$ be any $m \times n$ matrices.  The row space of $A$ is orthogonal to the null space of $A$.

Proof: Suppose $\mathbf{x}$ is a vector in the nullspace. Then $A \mathbf{x} = 0$. This equation can be considered as rows of $A$ multiplying $\mathbf{x}$:

$A\mathbf{x}$ = $\left[\begin{array}{ccc}\cdots & row_1 & \cdots \\\cdots & row_2 & \cdots \\ \phantom{a} & \phantom{a} & \phantom{a} \\ \cdots & row_m & \cdots \\ \end{array}\right]\left[\begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \\ \end{array} \right]$ = $\left[\begin{array}{c} 0 \\ 0 \\ \vdots \\ 0 \\ \end{array} \right]$.

We can clearly see that a dot product of row 1 and $\mathbf{x}$ is zero, a dot product of row 2 and $\mathbf{x}$ is zero, and so on. This means that the dot product of every row of $A$ and $\mathbf{x}$ is zero, which means every row of $A$ is orthogonal to $\mathbf{x}$, which completes the proof.

Corrolary 1
The column space of $A$ is orthogonal to the left null space of $A$.

Proof: The proof is easy, we just need to consider the transpose of the matrix $A$.

Going back to our original problem, when the vector $\mathbf{b}$ is in the column space of $A$, the vector will be orthogonal to any vector in the left null space of $A$.

Thus, we have the following solvability condition:

Solvability condition of $A \mathbf{x} = \mathbf{b}$ for any matrices $A$
The linear system $A\mathbf{x}=\mathbf{b}$ has a solution if the dot product of $\mathbf{b}$ and $\mathbf{y}_i$ is zero, where $\mathbf{y}_i$ is the base vector of the left null space of $A$.

Remark, the above solvability condition is closely related to the linear version of Fredholm solvability condition.

Reference
[1] Strang, G., “Linear Algebra and Its Applications,” Thomson Brooks/Cole

Categories: Justincase Tags: , ,