Posts Tagged ‘linear algebra’

Notes on diagonalizable linear operators

May 11, 2011 3 comments

Okay, I have a few things in my mind about this topic and I don’t want to lose them tomorrow or the day after. Therefore, I created this note. Hope it is useful to other people as well. Background of this note are as follows:

  1. Suppose A is an n \times n matrix and it is diagonalizable. Therefore, by definition, there is an invertible matrix P and a diagonal matrix D. However, I always forget the relationship between A and D, whether it is D=PAP^{-1} or D=P^{-1}AP. Generally, those two are no different, but we can choose P such that the column vectors of matrix P are the eigenvectors of matrix A, hence we have to know precisely which one it is, otherwise it gets wrong. Until now, when I forget about this, I always derive it and it takes some of my precious time (haha) but now after I have this, I can just open the Internet and look for this note.
  2. Point 1 above talks about diagonalizability of a matrix. But generally, we have a linear operator acting on a general vector field instead. A linear operator is called diagonalizable if there is a basis B such that the matrix representation of this linear operator is a diagonal matrix. It seems for me, diagonalizability of a linear operator and that of a matrix are talking about two different things. One is talking about how to find a basis, while the other is talking about how to find an invertible matrix. However, these two concepts turn out to be the same.

We begin with the following theorem.

Theorem 1 (Coordinate-change matrix)
Let E be a standard basis and B be another basis for an n-dimensional linear space V. Then there is an invertible matrix P such that for every v \in V

P (v)_B=(v)_E.                                                                                                          (1)

Matrix P is called the transition or coordinate-change matrix from the basis B to the basis E. Note that (v)_B and (v)_E are the coordinates of vector v with respect to basis B and E respectively. In fact, the column vectors of matrix P are the coordinates of the basis vectors of B with respect to the standard bases E (i.e. if B=\{ v_1,v_2,\dots,v_n \} then P=[(v_1)_E (v_2)_E \dots (v_n)_E] ).

We won’t prove the theorem as it can be found in any linear algebra textbook. However, using this theorem we can relate the diagonalizability of a linear operator and that of a matrix.Suppose T is a diagonalizable linear operator on an n-dimensional linear space V. Take x \in V then let y be y=T(x). Suppose E is a standard basis in V and B is a basis in V that consists of independent eigenvectors of T.

Then we have the following.

(i) (y)_E = [T(x)]_E = [T]_E (x)_E = A (x)_E, where A is a representation matrix of T with respect to standard basis E.
(ii) (y)_B = [T(x)]_B = [T]_B (x)_B = D (x)_B, where D is a representation matrix of T with respect to basis B. We also know that D is a diagonal matrix.

We want to show that using the theorem 1, D=PAP^{-1} or D=P^{-1}AP. From equation (1) we get:

P (y)_B = (y)_E = A (x)_E = A P (x)_B.

Hence, we have (y)_B = P^{-1}AP (x)_B. While from point (ii) we have (y)_B = D (x)_B. As a conclusion, we have,

D = P^{-1} A P.

We have derived that in fact, D = P^{-1} A P is the right one. We also derived that the invertible matrix P has such a close relationship with the basis B as the column vectors of P are the basis vectors of B, which are the eigenvectors of the linear operator T. This conclude my note.


The solvability condition of A x = b when A is not a square matrix

March 19, 2010 5 comments

In an elementary linear algebra course, we learned the solvability condition of the linear system

A \mathbf{x} =\mathbf{ b}

where A is an n \times n matrix and \mathbf{x}, \mathbf{b} are n \times 1 vectors.

To mention a few, the followings are the solvability conditions (equivalent conditions) such that the linear system above has a solution:

  1. determinant of A is not zero
  2. rank of A is full
  3. all eigenvalues of A are non zero
  4. the vector columns (also rows) are independent
  5. A has an inverse
  6. et cetera

However, I would like to discuss the case when either (#) A is not a square matrix or
(##) the rank of A is not full (< $latex  n$).

There is actually a condition such that the linear system A \mathbf{x} = \mathbf{b} has a solution whenever we have either (#) or (##). The solvability condition is

\mathbf {b} \in Col(A).

The thing is this condition is not applicable to most problems I have ever had. For instances, I bumped into the following problem.

Let A, \mathbf{x}, \mathbf{b} be the following

A=\left(\begin{array}{cc}a_{11} & a_{21} \\a_{12} & a_{22} \\b_{1} &b_{2} \\\end{array}\right)\mathbf{x} = (l_1, l_2)^T and \mathbf{b}=(-a_{11},-a_{22},0)^T. I want to seek a condition such that the linear system A \mathbf{x}=\mathbf{b} is solvable for l_1,l_2. I want to find a condition for a_{ij} and b_i such that there is a solution of l_1, l_2.

The statement \mathbf{b} \in Col(A) did not help me at that time and I look for another implication for this statement that is applicable enough to my problem.

Fundamental Theorem of Orthogonality (see [1])
Let A be any m \times n matrices.  The row space of A is orthogonal to the null space of A.

Proof: Suppose \mathbf{x} is a vector in the nullspace. Then A \mathbf{x} = 0. This equation can be considered as rows of A multiplying \mathbf{x}:

A\mathbf{x} = \left[\begin{array}{ccc}\cdots & row_1 & \cdots \\\cdots & row_2 & \cdots \\ \phantom{a}  & \phantom{a} & \phantom{a} \\ \cdots & row_m & \cdots \\ \end{array}\right]\left[\begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \\ \end{array} \right] = \left[\begin{array}{c} 0 \\ 0 \\ \vdots \\ 0 \\ \end{array}  \right].

We can clearly see that a dot product of row 1 and \mathbf{x} is zero, a dot product of row 2 and \mathbf{x} is zero, and so on. This means that the dot product of every row of A and \mathbf{x} is zero, which means every row of A is orthogonal to \mathbf{x}, which completes the proof.

Corrolary 1
The column space of A is orthogonal to the left null space of A.

Proof: The proof is easy, we just need to consider the transpose of the matrix A.

Going back to our original problem, when the vector \mathbf{b} is in the column space of A, the vector will be orthogonal to any vector in the left null space of A.

Thus, we have the following solvability condition:

Solvability condition of A \mathbf{x} = \mathbf{b} for any matrices A
The linear system A\mathbf{x}=\mathbf{b} has a solution if the dot product of \mathbf{b} and \mathbf{y}_i is zero, where \mathbf{y}_i is the base vector of the left null space of A.

Remark, the above solvability condition is closely related to the linear version of Fredholm solvability condition.

[1] Strang, G., “Linear Algebra and Its Applications,” Thomson Brooks/Cole

Categories: Justincase Tags: , ,