There is a variety of problems where one searches for a vector v such that the squared sum of the lengths of projections of points xi onto that vector is maximized or minimized:
vminE(v)s.t.v⊺v=i=1∑n⟨xi,v⟩2=i=1∑nxi⊺v⋅xi⊺v=i=1∑nv⊺xi⋅xi⊺v=v⊺X⊺Xv=1
This can be written as a Lagrange function by introducing a Lagrangian multiplier λ for the constraint:
L(v,λ)=v⊺X⊺Xv−λ(1−v⊺v)
Optimization of the original problem is equivalent to finding a solution to the following equation system:
∇v,λL(v,λ)=0⇔{X⊺Xv=λvv⊺v=1
The first line of this equation tells us that v must be an eigenvector of X⊺X with eigenvalue λ. Let’s call those ve and λe. Plugging this solution back into our original problem (and remembering that the eigenvector is not necessarily unit length) we obtain:
E(∣ve∣ve)=ve⊺veve⊺X⊺Xve=ve⊺veve⊺λve=λ
Depending on whether we want to obtain a minimum or a maximum we can choose the eigenvector with the smallest or largest eigenvalue respectively.