Information about Dual Problem

In linear programming, the primary problem and the dual problem are complementary. A solution to either one determines a solution to both.

Background

Linear programming problems are optimization problems in which the objective function and the constraints are all linear.

Problem statement in the linear case

In the primal problem, the objective function is a linear combination of n variables. There are m constraints, each of which places an upper bound on a linear combination of the n variables. The goal is to maximize the value of the objective function subject to the constraints. A solution is a vector (a list) of n values that achieves the maximum value for the objective function.

In the dual problem, the objective function is a linear combination of the m values that are the limits in the m constraints from the primal problem. There are n dual constraints, each of which places a lower bound on a linear combination of m dual variables.

Duality principle

In optimization theory, the duality principle states that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem.

Relationship between the primal problem and the dual problem

In the linear case, in the primal problem, from each sub-optimal point that satisfies all the constraints, there is a direction or subspace of directions to move that increases the objective function. Moving in any such direction is said to remove slack between the candidate solution and one or more constraints. An infeasible value of the candidate solution is one that exceeds one or more of the constraints.

In the dual problem, the dual vector multiplies the constants that determine the positions of the constraints in the primal. Varying the dual vector in the dual problem is equivalent to revising the upper bounds in the primal problem. The lowest upper bound is sought. That is, the dual vector is minimized in order to remove slack between the candidate positions of the constraints and the actual optimum. An infeasible value of the dual vector is one that is too low. It sets the candidate positions of one or more of the constraints in a position that excludes the actual optimum.

This intuition is made formal by the equations under Duality in the article on linear programming.

The non-linear case

In non-linear programming, the constraints are not necessarily linear. Nonetheless, many of the same principles apply.

In order to ensure that a non-linear problem will have a unique global maximum, it is conventional to resort to simplifying criteria such as convexity. If the curvature of the constraints and of the objective function is such that the feasible region is always convex, then there will still be a unique global optimum.

This is the significance of the Karush-Kuhn-Tucker conditions. They provide sufficient conditions based upon convexity for identifying non-linear programming problems that do have a unique global optimum. There are additional conditions that are necessary so that it will be possible to define the direction to an optimal solution. An optimal solution is one that is a local optimum, but possibly not a global optimum.

Historical remarks

Concerning the origins of the duality theorem, Saul Gass cites Nering and Tucker, 1993. In his foreword to that book, George Dantzig writes that the duality theorem was a conjecture of von Neumann, and that rigorous proofs were first published in 1948 by Albert W. Tucker and his group. The article also says that Dantzig independently stated and proved the duality theorem in an internal Air Force report in the same year.

References

• Dantzig, G.B., 1963. Linear Programming and Extensions, Princeton University Press, Princeton, NJ.
• Gass, Saul I., IFORS' Operational Research Hall of Fame: Albert William Tucker, International Transactions in Operational Research, Volume 11 Issue 2, Page 239 - March 2004, doi:10.1111/j.1475-3995.2004.00455.x. http://www.blackwell-synergy.com/doi/abs/10.1111/j.1475-3995.2004.00455.x
• Gass, Saul I., IFORS' Operational Research Hall of Fame: George B. Dantzig, International Transactions in Operational Research, Volume 10 Issue 2 Page 191 - March 2003, doi:10.1111/1475-3995.00403. http://www.blackwell-synergy.com/doi/abs/10.1111/1475-3995.00403
• Nering, E.D and Tucker, A.W., 1993, Linear Programming and Related Problems, Academic Press, Boston, MA.