how to interpret the SVD of a data matrix ASo I understand the proofs behind Singular Value Decomposition but I'm having trouble interpreting it in the context of a real world problem.Specifically, If I'm given an m × n data matrix A, where we have m training examples and n features collected for each example, I'm having trouble understanding the meaning behind A v j = σ j u j where L A (left multiplication by A) is our linear transformation and β = { v 1 , v 2 , . . . , v n } is an orthonormal basis for F n and γ = { u 1 , u 2 , . . . , u m } is an orthonormal basis for F m .From reading various posts and articles, the idea seems to be a larger σ indicates more variation in the data along that vector j while a smaller variation in a certain direction j is captured with a smaller σ.However, when we are looking at σ i'm not sure why we care about what A is doing. After all, this relationship would be great if I wanted to see what A does when it acts on a orthonormal basis β but A is just a data matrix so i'm not sure how to interpret the range of a data matrix or what types of transformations A is making when presented a vector x to 'do' left multiplication on.

Question

how to interpret the SVD of a data matrix ASo I understand the proofs behind Singular Value Decomposition but I&#039;m having trouble interpreting it in the context of a real world problem.Specifically, If I&#039;m given an   m  ×  n data matrix A, where we have m training examples and n features collected for each example, I&#039;m having trouble understanding the meaning behind   A      v    j    =      σ    j        u    j   where L          A   (left multiplication by A) is our linear transformation and   β  =  {      v    1    ,      v    2    ,  .  .  .  ,      v    n    } is an orthonormal basis for F          n   and   γ  =  {      u    1    ,      u    2    ,  .  .  .  ,      u    m    } is an orthonormal basis for F          m  .From reading various posts and articles, the idea seems to be a larger   σ indicates more variation in the data along that vector           j   while a smaller variation in a certain direction           j   is captured with a smaller   σ.However, when we are looking at   σ i&#039;m not sure why we care about what           A   is doing. After all, this relationship would be great if I wanted to see what           A   does when it acts on a orthonormal basis   β but A is just a data matrix so i&#039;m not sure how to interpret the range of a data matrix or what types of transformations A is making when presented a vector x to &#039;do&#039; left multiplication on.

Blaine Foster · Accepted Answer

Your data sets are the rows of A. Thus you get the kth sample by computing       e    k    T    A. Using the SVD this is also      e    k    T    A  =      ∑          j      =      1        n              σ        j    (      e    k    T              u        j    )              v        j    T  You can reduce this sum to the leading d terms with the largest d singular values to get a good approximation of the data, which means that your data vectors are all close to the subspace spanned by             v        1    ,  …  ,            v        d   for some suitably chosen d

how to interpret the SVD of a data matrix A So I understand the proofs behind Singular Value Decomp

Answered question

Answer & Explanation

New Questions in Pre-Algebra