Condition for existence of an orthonormal matrix whose column space is orthogonal to the column space of another matrixWhile I was reading a statistics paper, I came across one statement that I don't understand (I just have basic linear algebra knowledge).Assume (in the context of regressions), we have a n × p data matrix X, assuming that X is invertible and n&gt;p. The paper states" U ∈ R n × p is an orthonormal matrix whose column space is orthogonal to that of X s.t. U T X = 0": such matrix exists if n ≥ 2 p. I don't understand where the last statement comes from.I know that the nullspace of X has dimension n−rank(X)=n−p in full rank case and U is the orthonormal basis of the null space of X. But I don't get the link why U only exists, if n ≥ p + r a n k ( X ), i.e. n ≥ 2 p.

Question

Condition for existence of an orthonormal matrix whose column space is orthogonal to the column space of another matrixWhile I was reading a statistics paper, I came across one statement that I don&#039;t understand (I just have basic linear algebra knowledge).Assume (in the context of regressions), we have a   n  ×  p data matrix X, assuming that X is invertible and n&amp;gt;p. The paper states&quot;  U  ∈            R              n      ×      p       is an orthonormal matrix whose column space is orthogonal to that of X s.t.       U    T    X  =  0&quot;: such matrix exists if   n  ≥  2  p. I don&#039;t understand where the last statement comes from.I know that the nullspace of X has dimension n−rank(X)=n−p in full rank case and U is the orthonormal basis of the null space of X. But I don&#039;t get the link why U only exists, if   n  ≥  p  +  r  a  n  k  (  X  ), i.e.   n  ≥  2  p.

billyfcash5n · Accepted Answer

Note: it is not standard to refer to the matrix X as &quot;invertible&quot; unless X is a square matrix. Presumably, &quot;X is invertible&quot; refers in this case to the fact that X has full rank. In this case, because X is   n  ×  p with n&amp;gt;p, this means that X has rank p (i.e. has &quot;full column rank&quot;).As you say, &quot;  U  ∈            R              n      ×      p       is an orthonormal matrix whose column space is orthogonal to that of X&quot;. The fact that the column space is orthogonal to that of X is equivalent to the statement that       U    T    X  =  0. Because U is   n  ×  p with orthonormal columns, the dimension of its column space is p. Because the column space of U is orthogonal to that of X, the column space of U must be a subspace of the orthogonal complement to the column space of X. The column space of X is a p-dimensional subspace of             R        n  , which means that its orthogonal complement has dimension n−p.Putting all this together leads us to the following conclusion:  col  ⁡  (  U  )  ⊆  col  ⁡  (  X      )    ⊥      ⟹      dim  ⁡  (  col  ⁡  (  U  )  )  ≤  dim  ⁡  (  col  ⁡  (  X      )    ⊥    )    ⟹      p  ≤  n  −  p    ⟹      2  p  ≤  n  .

Condition for existence of an orthonormal matrix whose column space is orthogonal to the column spac

Answered question

Answer & Explanation

New Questions in Pre-Algebra