Hamiltonian (control theory)

In control theory, the Hamiltonian is a function that plays a central role in optimal control problems. It is a scalar function of the state variables, control inputs, costate variables (also known as adjoint variables or Lagrange multipliers), and possibly time. The Hamiltonian provides a convenient way to formulate and analyze optimal control problems, especially using Pontryagin's Minimum Principle (also known as Pontryagin's Maximum Principle, depending on the convention).

The Hamiltonian, often denoted by H, is defined as follows:

H(x(t), u(t), λ(t), t) = L(x(t), u(t), t) + λ(t)^T f(x(t), u(t), t)

where:

• x(t) is the state vector at time t.
• u(t) is the control input vector at time t.
• λ(t) is the costate vector (a vector of Lagrange multipliers) at time t.
• L(x(t), u(t), t) is the running cost (also known as the Lagrangian) incurred at time t. This represents the instantaneous cost of the system's state and control.
• f(x(t), u(t), t) represents the state dynamics, describing how the state evolves over time as a function of the current state and control input: dx(t)/dt = f(x(t), u(t), t).
• T denotes the transpose of the vector.

The costate vector λ(t) can be interpreted as the sensitivity of the optimal cost with respect to changes in the state x(t). It essentially represents the shadow price or marginal value of being in a particular state at a particular time.

Role in Pontryagin's Minimum Principle:

The Hamiltonian is a key component of Pontryagin's Minimum Principle, which provides necessary conditions for optimality in control problems. According to the principle, if u(t)* is an optimal control that minimizes the cost functional, then there exists a costate vector λ(t) such that:

1. The optimal control u(t)* minimizes the Hamiltonian pointwise in time:

   H(x(t), u(t), λ(t), t) ≤ H(x(t), u(t), λ(t)*, t) for all admissible u(t)

2. The costate vector satisfies the costate equation (adjoint equation):

   dλ(t)/dt = -∂H/∂x(t)

3. The state vector satisfies the state equation:

   dx(t)/dt = ∂H/∂λ(t) = f(x(t), u(t), t)

4. Appropriate boundary conditions are met on the state and costate vectors, depending on the specific problem (e.g., initial state constraint, final state constraint, or transversality conditions).

Significance:

The Hamiltonian formulation allows complex optimal control problems to be addressed systematically. By formulating the problem in terms of the Hamiltonian and applying Pontryagin's Minimum Principle, one can derive a set of differential equations (the state and costate equations) and an algebraic condition for the optimal control. Solving these equations, often analytically or numerically, provides the optimal state trajectory and control input. The Hamiltonian is also used in numerical optimization techniques like indirect methods for solving optimal control problems.