I need help solving this following equation, a Lagrangian problem that I encountered during my studies in principal component analysis (PCA).One should maximize the variance with respect to the first principal componentFollowing is the case: We know that W = v 1 T S v 1 (eq 1) and that v 1 is an orthonormal basis.Therefore | | v 1 | | 2 = v 1 T v 1 = 1 (eq 2). The maximization of W under this constraint can be done by introducing the Lagrangian multiplier λ and maximizing the Lagrangian: L = W + λ ( 1 − | | v 1 | | 2 ) = v 1 T ( S − λ I ) v 1 + λ (eq 3)So expanding W using eq. 1, and the laws of distributivity we end up with eq 3.Next, taking the derivative with respect to vector and λ we obtain: ∂ ∂ λ L = 1 − v 1 T v 1 = 0 ∇ V 1 L = ( S − λ I ) v 1 = 0This can later be shown to be an eigenvalue problem. Can anyone explain to me why the derivatives look like that? I can't seem to figure it out.

Question

I need help solving this following equation, a Lagrangian problem that I encountered during my studies in principal component analysis (PCA).One should maximize the variance with respect to the first principal componentFollowing is the case: We know that   W  =            v        1    T              S      v        1   (eq 1) and that       v    1   is an orthonormal basis.Therefore       |        |                      v            1            |              |        2    =            v        1    T              v        1    =  1 (eq 2). The maximization of   W under this constraint can be done by introducing the Lagrangian multiplier   λ and maximizing the Lagrangian:      L    =  W  +  λ  (  1  −      |        |              v        1        |              |        2    )  =            v        1    T    (      S    −  λ      I    )            v        1    +  λ (eq 3)So expanding   W using eq. 1, and the laws of distributivity we end up with eq 3.Next, taking the derivative with respect to vector and   λ we obtain:      ∂          ∂      λ            L    =  1  −            v        1    T              v        1    =  0      ∇                  V                              1            L    =  (      S    −  λ      I    )            v        1    =  0This can later be shown to be an eigenvalue problem. Can anyone explain to me why the derivatives look like that? I can&#039;t seem to figure it out.

Josie Stephenson · Accepted Answer

I believe I found a solution, to whom this may be interesting.First partial differentiation is straight forward,   λ is a constan. I found a slightly different approach, remember the inner product is = 1.We know the fact that:  λ      ∂          ∂                        v                1        T              (            v        1    T              v        1    )  =  2  λ            v        1  The second one, uses the fact that      ∂          ∂                        v                1              (            v        1    T    (      S    −  λ      I    )            v        1    )  =  2  (      S    −  λ      I    )            v        1  Hence the partial differentiation yields:      ∇                            v                1                  L    =  2  (      S    −  λ      I    )            v        1    −  2  λ            v        1  The optimization problem then becomes:            ∂              L                    ∂                                    v                    1                      =  (      S    −  λ      I    )            v        1    −  λ            v        1    =  0Normalization of             v        1   and rewriting of the optimization problem yields the eigenvalue problem:            S      v        1    =  λ            v        1

I need help solving this following equation, a Lagrangian problem that I encountered during my studi

Answered question

Answer & Explanation

New Questions in High school geometry