Suppose that there is a positive definite matrix <mrow class="MJX-TeXAtom-ORD"> <mi mathvar

Rebecca Villa

Rebecca Villa

Answered question

2022-07-10

Suppose that there is a positive definite matrix A R n × n , and a vector b R n , then minimization of quadratic functions with linear terms can be done in closed form as
arg min x R n ( 1 2 x T A x b T x ) = A 1 b

I met this in a machine learning book. However, the book didn't provide a proof. I wonder why this can be well-formed. Hope that someone can help me with it. I find that many machine learning books like to skip all of the proofs, which made me uncomfortable.

Answer & Explanation

Melina Richard

Melina Richard

Beginner2022-07-11Added 14 answers

One way of proving this is to "complete the square":
1 2 x A x b x = 1 2 ( ( x A 1 b ) A ( x A 1 b ) b A 1 b )   .
.Because   A   is positive definite this is never less than   1 2 b A 1 b  , and it attains that value when   x = A 1 b  .

Do you have a similar question?

Recalculate according to your conditions!

New Questions in Multivariable calculus

Ask your question.
Get an expert answer.

Let our experts help you. Answer in as fast as 15 minutes.

Didn't find what you were looking for?