A visual explanation of the Fenchel-Rockafellar Theorem

Posted by Philipp Wacker on September 28, 2018

Visualization of \(\inf_{x\in E}\{\phi(x) + \psi(x)\}\)

We can write \(\inf_{x\in E}\{\phi(x) + \psi(x)\} = \inf_{x\in E}\{\phi(x) - (-\psi(x))\}\). The absolute value of \(\color{mygreen}{\phi(x) - (-\psi(x))}\) is the length of the vertical vector connecting the graph of \(\psi\) with the graph of \(\phi\). Its sign is positive if the vector points up (i.e. if \(\phi(x) > \psi(x)\)) and down vice versa.

This means that \(\color{mymaroon}{\inf_{x\in E}\{\phi(x) + \psi(x)\}}\) is the infimum over all possible values for this signed vector length.

Change the value of \(x\) with this slider to convince yourself that the minimum of \(\phi(x)+\psi(x)\) is at \(x \approx 0.68\).

\(x = ~\)0

Plot of \(\color{blue}\phi(x)\) and \(\color{red}-\psi(x)\).
Plot of \(\color{blue}\phi(x)-(-\psi(x))\) .

Visualization of
\(\sup_{f\in E^*}\{-\phi^*(f) -\psi^*(-f)\}\)

From before we know that \(-\phi^*(f)\) is the smallest signed vector length between the graph of \(\phi\) and \(\langle f, \cdot \rangle\).

Plot of \(\color{blue}\phi(x)\), \(\color{red}x\mapsto \langle f, x\rangle\) and \(\color{green}-\phi^*(f)\) .

In the same way, \(-\psi^*(-f)\) is the smallest signed vector length between the graph of \(-\psi\) and \(\langle f, \cdot \rangle\).

Plot of \(\color{blue}-\psi(x)\), \(\color{red}x\mapsto \langle f, x\rangle\) and \(\color{green}-\psi^*(-f)\) .

Hence, \(\color{mymaroon}{\sup_{f\in E^*}\{-\phi^*(f) -\psi^*(-f)\}}\) is the highest possible vector length resulting from adding the two together.

\(f = ~\)0

Plot of \(\color{blue}\phi(x)\), \(\color{darkblue}-\psi(x)\), \(\color{red}x\mapsto \langle f, x\rangle\) and conjugate quantities \(\color{green}-\phi^*(f)\), \(\color{darkgreen}-\psi^*(-f)\).
Plot of \(\color{grey}-\phi^*(f)-\psi^*(-f)\) .

Now compare plots of \(\phi(x) + \psi(x)\) and \(-\phi^*(f)-\psi^*(-f)\) to see that the infimum of the former is equal to the supremum of the latter. The reason why this is so is not obvious. It is explained in the proof of the Fenchel–Rockafellar theorem.

Plot of \(\color{blue}\phi(x)+\psi(x)\).
Plot of \(\color{blue}-\phi^*(f)-\psi^*(-f)\).