Convex Functions and Jensen's Inequality

A real-valued function $f$ is convex on an interval $I$ if and only if

\begin{displaymath} f(ta +(1-t)b) \le tf(a)+(1-t)f(b) \end{displaymath} (1)

for all $a,b\in I$ and $0\le t\le 1$. This just says that a function is convex if the graph of the function lies below its secants. See pages 2 through 5 of Bjorn Poonen's paper, distributed at his talk on inequalities, for a discussion of convex functions and inequalities for convex functions. A number of common functions that are convex are also listed. Among those listed are $-\ln x $ on $(0,\infty)$, $ -\sin x$ on $[0,\pi]$, $-\cos x$ on $[-\pi/2,\pi/2]$ and $\tan x$ on $[0,\pi/2]$. To avoid the negative signs a complementary concept is defined. A real-valued function $f$ is concave on an interval $I$ if and only if
\begin{displaymath} f(ta +(1-t)b) \ge tf(a)+(1-t)f(b) \end{displaymath} (2)

for all $a,b\in I$ and $0\le t\le 1$. Therefore $f$ is convex iff $-f$ is concave. If you are familiar with derivatives then the following theorem about twice differentiable functions provides a way of telling if such a function is convex.

Theorem 3   If $f^{\prime\prime}(x) \ge 0$ for all $x \in I$ , then f is convex on I.

Inequality ( 1) can be generalized to a convex function $f$ with three variables $x_1,x_2,x_3$ with weights $t_1,t_2,t_3$, respectively, such that $t_1+t_2+t_3=1$. Note that $t_2+t_3=1-t_1$. In this manner the three variable case can be transformed into the two variable case as follows.

\begin{eqnarray*} f(t_1x_1+t_2x_2+t_3x_3) & = & f\left(t_1x_1+ (1-t_1)\frac{t_2x... ...}{t_2+t_3}f(x_3) \right)\\ & = & t_1f(x_1)+t_2f(x_2)+t_3f(x_3). \end{eqnarray*}



This process can be continued to produce an $n$ variable version which is due to J.L.W.V. Jensen. It can be easily proved by mathematical induction using the above technique. Write your own proof and compare with the one given here. It will give you some good practice manipulating sigma notation.

Theorem 4 (Jensen's Inequality 1906)   Let f be a convex function on the interval I. If $x_1, x_2,\dots,a_n \in I$ and $t_1,t_2,\dots,t_n$ are nonnegative real numbers such that $t_1+t_2+\dots+t_n=1$, then

\begin{displaymath}f(\sum_{i=1}^{n}t_ix_i) \le \sum_{i=1}^{n}t_if(x_i).\end{displaymath}

Proof by induction: The case for $n=2$ is true by the definition of convex. Assume the relation holds for $n$, then we have

\begin{eqnarray*} f\left(\sum_{i=1}^{n+1}t_ix_i\right) &=& f\left(\sum_{i=1}^{n}... ...t_if(x_i) +t_{n+1}f(x_{n+1})\\ & = & \sum_{i=1}^{n+1}t_if(x_i). \end{eqnarray*}



Thus showing that the assumption implies that the relation holds for $n+1$ and by the principle of Mathematical Induction holds for all natural numbers.
An easy consequence of Jensen's theorem is the following proof of the arithmetic mean-geometric mean inequality. (Problem 13 from Bjorn's paper)

Theorem 5 (AM-GM Inequality)   If $x_1,x_2,\dots,x_n \ge 0 $ then

\begin{displaymath}\frac{x_1+x_2+\cdots+x_n}{n} \ge \sqrt[n]{x_1x_2\cdots x_n}.\end{displaymath}

Proof. Since $-\ln x $ is convex then $\ln x$ is concave. By Jensen's theorem we have

\begin{eqnarray*} \ln\left(\frac{x_1+x_2+\cdots+x_n}{n}\right) & \ge &\frac{ \ln... ..._2 \cdots x_n) \\ & = & \ln[(x_1 x_2 \cdots x_n)^{\frac{1}{n}}] \end{eqnarray*}



Since $\ln x$ is monotonic increasing ( $f^{\prime}(x) = \frac{1}{x}>0$) for $x>0$ we have
$ \frac{x_1+x_2+\cdots+x_n}{n} \ge \sqrt[n]{x_1x_2\cdots x_n}.$
The proof of Jensen's Inequality does not address the specification of the cases of equality. It can be shown that strict inequality exists unless all of the $x_i$ are equal or $f$ is linear on an interval containing all of the $x_i$.