Questions tagged [optimal-control]
Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)
1,067 questions
2
votes
0
answers
22
views
Solution to a Constrained Optimal Control Problem: Is $v(t) = S/T$ Optimal
The problem is stated as:
$\min_{v}\int^T_0f(v(t),t)dt$
Subject to the following constraints:
$s'(t)=v(t)$,
$s(0)=0$,
$s(T)=S$,
$v_{\min}\le v(t),S/T\le v_{\max}$
where:
$T$ and $S$ are given ...
- 23
4
votes
1
answer
60
views
Are the motion equations of an optimal control problem geodesics on a manifold?
Let us consider a Lagrangian system for which the equations of motion come from Hamilton's principle and are such that the control variable $\tau$ equals the equations of motion of a free system, ...
- 534
0
votes
0
answers
22
views
Adjoint state coming from Pontryagin Maximum Principle is normal to the attainable set - Reference Search
Given an optimal control problem
$$
\begin{cases}
\min_{u} &\int_s^T \ell(x(t), u(t)) \, dt + g(x(T))\\
\text{s.t.} & \dot{x}(t) = f(x(t),u(t))\\
& x(s) = x_0\\
& u \in \mathcal{A}\...
- 2,795
2
votes
1
answer
77
views
Optimal betting strategy to reach 8 points before 0 with success probability 0.4
I am studying a variant of the gambler’s ruin problem and would like help formulating and solving it optimally.
A student starts with an initial capital of 3 points.
To pass the exam, he must reach 8 ...
- 54
0
votes
0
answers
29
views
Continuity of an optimal stopping value with discontinuous gain function.
I am trying to approach this homework on optimal stopping. Suppose we have an optimal stopping problem where we observe the process $$dm_t = \frac{1}{1+t}dW_t,$$
where $W_t$ is a standard Brownian ...
- 117
0
votes
1
answer
51
views
How to apply Pontryagin's principle to a certain minimum-time problem
Background
I'm trying to solve an optimal control problem using Pontryagin's Principle. The problem involves finding a control function that minimizes the time from a given initial state to a given ...
- 171
0
votes
0
answers
84
views
Why Is it Difficult to Ensure Stability for RL-based Control Algorithms?
For context, I am a layman, although I do have some background in basic college differential equations and linear algebra.
I read that one of the drawbacks of control methods based on reinforcement ...
0
votes
1
answer
87
views
Optimal Control w/ Control Equality Constraints
Let's say I have an optimal control problem of the following for:
\begin{align*}
&\max_{\vec{u}}\int_a^bf(\vec{x}(t),\vec{u}(t))\,dt\\[10pt]
\text{subject to:}&\qquad \frac{d\vec{x}}{dt}(t) = \...
- 10.1k
12
votes
2
answers
998
views
Proof of shortest path avoiding ball
I have read in a number of places that the shortest path between two points $a,b\in \mathbb{R}^2$ that avoids a disk $D$ between them (by "between"
I mean the disk intersects the line $a-b$) ...
1
vote
0
answers
87
views
Minimization with implicit dependency over function spaces
Technically, I am working on an optimal control problem. However, through some trickery, I managed to eliminate the dynamics. What I am left with is the following minimization problem:
$$ \min_{g \in ...
- 66
0
votes
0
answers
48
views
Stochastic Optimal Control: Example of problem with no optimal control
I have a problem with the proof of Exercise 4.17 iv) in the following script: https://www.maths.ed.ac.uk/~dsiska/LecNotesSCDAA.pdf
Here I state the Exercise as in this script together with the ...
- 31
0
votes
1
answer
46
views
Finding an optimal interpolation subject to some boundary constraints
Let $s\in \left(0, 1\right)$, and consider the task of finding the 'nicest-possible' function $f$ such that
$f(0) = 0, f^\prime (0) = s$, and
$f(1) = 1, f^\prime (1) = 0$,
where 'niceness' is framed ...
- 11.1k
1
vote
1
answer
132
views
Why does a Luenberger observer require observability?
I have the following misunderstanding regarding observability and observer.
Observability of a linear dynamical system is the same as the "recoverability" of the initial condition $x(0)$. A ...
- 8,711
7
votes
0
answers
337
views
Kalman Filter with correlated measurement noise derivation
I have made great efforts on the derivation, and the results are really close but I am still missing the last step. If someone can help that'd be great!
Problem setup
Consider this modified Kalman ...
- 479
3
votes
1
answer
136
views
Why is that in some formulations of discrete-time optimal control, the state is also a variable?
Consider the formulation of the discrete-time LQR optimal control as shown in the screenshot below (source)
Why are $x_1, x_2, \ldots$ the variables as well?
Since all the $x_k$ depends on $u_k$ and ...
- 8,711
2
votes
1
answer
123
views
Clarification of Battles of Extinction
I am working through chapter 3 of Rufus Isaacs's work on differential games which is devoted to discrete games. I am stuck trying to understand his section 3.3 Battles of Extinction game where there ...
- 479
0
votes
0
answers
47
views
Interpretation of adjoint in optimal control problem
This is a follow-up question to my previous question.
I'm considering the following optimal control problem where the idea is to maximize the state $x$ using some control $u$, and we know the dynamics ...
- 10.1k
4
votes
2
answers
157
views
Unusual Optimal Control Problem - Question about interpretation of $\lambda$
I'm trying to understand what seems like a very basic optimal control problem, but I'm getting two solutions that appear to be different and I'd like some help with clearing up the discrepancy. The ...
- 10.1k
0
votes
0
answers
40
views
What is the relation between the Maximum Principle and strong duality?
Consider an optimal control problem with a control $u\in U$ and state $x$. We want to maximize:
$$
\int_0^1 J(t,x(t),u(t)) dt
$$
The law of motion is $x'(t)=y(t)$. The end-points of $x(0),x(1)$ are ...
- 321
3
votes
0
answers
71
views
Optimal control problem with free initial state
Consider an optimal control problem with a control $u\in U$ and states $x,y$. We want to maximize:
$$
\int_0^1 J(t,x(t),y(t),u(t)) dt
$$
The laws of motion are is simply $x'(t)=y(t)$ and $y'(t)=u(t)$.
...
- 321
2
votes
0
answers
68
views
Lower bounds for length of bounded curvature path
It is known that the shortest path of bounded (pointwise) curvature between two points in $\mathbb{R}^2$ with specified initial and final tangent vectors is the concatenation of circle arcs and line ...
3
votes
1
answer
118
views
Riccati equation with singular coefficients
In a problem I am looking at, I encounter a matrix Riccati equation of the following form:
$$\dot{\bf W} (t) - {\bf W} (t)\, {\bf U} \, {\bf W} (t) - {\bf V}^\top {\bf W} (t) - {\bf W} (t) \, {\bf V} -...
- 61
1
vote
1
answer
162
views
How can I linearize a state space model about non-fixed points?
I have an adequate understanding of state-space models. I've stabilized an inverted pendulum cart robot with a linear quadratic regulator by modeling the system, finding the Jacobian of the nonlinear ...
- 331
0
votes
0
answers
42
views
Maximum Principle with time-varying control set
Consider an optimal control problem with a control $u\in U(t)$ and state $x$. We want to maximize:
$$
\int_0^1 J(t,x(t),u(t)) dt
$$
where the end-points of $x(t)$ are fixed. The law of motion is ...
- 321
2
votes
3
answers
113
views
Minimum energy function with average $0$ and defined boundary conditions
What continuous function $f$ solves the following minimization problem?
$$ \begin{array}{ll} \underset{f \in C([0,1])}{\text{minimize}} & \displaystyle\int\limits_0^1 f(x)^2 {\rm d}x \\ \text{...
1
vote
1
answer
181
views
What is the exact connection between PID and state feedback controllers?
Suppose we are dealing with a control problem where the reference trajectory is $0$, then the PID controller is a function,
$$u = K_1 x + K_2 \dot x + K_3 \int_0^t x dt$$
where $x$ is your state.
But ...
- 8,711
0
votes
1
answer
93
views
Unscented Kalman Filter: Weighted Average fails on Heading state
The UKF I implement
I am currently implementing an unscented Kalman filter (UKF) to estimate the 2D Pose and speed and the wheels steering angle $(x,y,heading, velocity, steeringAngle)$ of a car. I ...
- 1
1
vote
1
answer
118
views
Mean Field Games Solving the Couple HJB system
The mean field game for a given minimization problem leads to a coupled system of HJB and Fokker-Planck equations. The HJB is forward in time, where as the FPK equations are backwards. How does one ...
- 511
1
vote
1
answer
126
views
Relaxing a binary constraint in an optimal control problem
Crossposted at Operations Research SE
I am attempting to optimize the operation of an electrical system that produces some amount of thermal power $P_t$ and keeps a temperature $x_t$ within a certain ...
- 11
1
vote
1
answer
69
views
stability analysis with more than 2 dynamic variables using eigenvalues
I have a system of 8 differential equations derived from an optimal control problem with 4 controls and 4 state variables. I get two interior steady state equilibria, and I want to see whether each is ...
- 21
2
votes
1
answer
101
views
Relationship between equilibrium points and controllable subspace of a linear system.
Suppose I have a discrete-time linear dynamical system $x_{t+1} = Ax_t + Bu_t$, with no constraints on $A$ and $B$, e.g. the system may not be controllable, A may not be stable. B may not be full ...
- 577
0
votes
0
answers
74
views
Some minimization problems related to optimal control
Motivation I have been interested in understanding the relationships between some frequency domain conditions, Riccati equations and matrix inequalities (such as the question asked here). I came to ...
- 413
9
votes
1
answer
208
views
Equivalence of Lyapunov equation, matrix inequality and algebraic Riccati equation versions of the Positive Real Lemma
My question is about the equivalence of three different versions of the positive real lemma.
I would like to set up the question by first stating the definition of
a positive real transfer function ...
- 413
2
votes
0
answers
112
views
Nonzero setpoint for LQR with/without steady-state input [closed]
Suppose I have a linear dynamical system $x_{t+1} = Ax_t + Bu_t$ that I'm trying to drive to a nonzero target state $x_{ss}$ (suppose that the system is controllable, or that $x_{ss}$ is in the ...
- 577
3
votes
1
answer
255
views
Controllable system with a state that cannot be maintained
I'm confused about this statement from Wikipedia:
Controllability does not mean that a reached state can be maintained,
merely that any state can be reached.
What is an example of a system that is ...
- 577
0
votes
0
answers
66
views
Checking the Maximum Principle where state constraints are present but slack: can I drop them?
I have an optimal control problem with a control $u\in [a,b]$, a state $x(t)$ and the law of motion $x'(t)=f(t,u(t))$ for some smooth $f$. The control is chosen to maximize:
$$
\int_0^1 J[t,x(t),u(t)] ...
- 321
1
vote
0
answers
52
views
Verify argument: I use multipliers on integral constraints in optimal control
I have an optimal control problem with a control $u\in [a,b]$, a state $x(t)$ and the law of motion $x'(t)=f(t,u(t))$ for some smooth $f$. I want to maximize:
$$
\int_0^1 J[t,x(t),u(t)] dt
$$
subject ...
- 321
0
votes
0
answers
41
views
Optimal control version of Legendre condition from calculus of variations?
Let $F$ be twice-continuously differentiable and consider a calculus of variations problem of maximizing the functional
$$
\int_0^1 F(t,x,x')dt
$$
over the space of twice-continuously differentiable $...
- 321
4
votes
3
answers
136
views
$P_{\infty}=R$ in steady-state Kalman Filter when transition and observation matrix = $I$
Considering a simple Kalman Filter
State update equation $x_t = x_{t-1} + w_t, w_t\sim N(0,Q)$
Observation equation $z_t = x_{t} + v_t, v_t\sim N(0,R)$
I'm curious, under what conditions, will we have ...
- 41
5
votes
1
answer
373
views
Kalman Filter with correlated observation noise and identity observation matrix
My Kalman Filter model is very simple, I have a identity transition matrix and identity observation matrix.
State transition: $x_t = x_{t-1} + v_t$ where $v_t \sim N(0,Q)$. The states kind of follow a ...
- 163
1
vote
0
answers
54
views
How to get the minimum time and the initial adjoint in control problem?
I am trying to implement a shooting algorithm to solve a atate-to-state transfer of two-level quantum system. The Hamiltonian of the system is given by
$$\hat{H} = \frac{u_x}{2}\hat{\sigma}_x + \frac{...
- 85
3
votes
1
answer
119
views
Finding the least time path through n ordered points with bound acceleration
Let's assume we have a body in $2$-dimensional space, and we want it to pass through a series of points in a certain order with some given starting velocity, and a constraint that it can only move ...
1
vote
0
answers
87
views
What does the Maximum Principle tell us when $\lambda'(t)=0$?
Consider an optimal control problem where $u(t)$ is the control and $x(t)$ and $y(t)$ are states. The law of motion for $x(t)$ is $x'(t)=u(t)$. The law of motion for $y(t)$ is $y'(t)=x(t)$. We are ...
- 321
3
votes
1
answer
150
views
Next steps for finding the extremal of a function that is independent to $t$ (Using $\dot{x} = \tan\chi$ substitution)
I am asked to find the extremal of the following function:
\begin{align*}
\int_0^1\frac{(1+\dot{x}^2)^\frac{1}{2}}{x} \: dt
\end{align*}
With the boundary conditions of $x(0)=0$ and $x(1)=\sqrt3$
...
- 33
1
vote
0
answers
75
views
Approximation of the Set of LQR feedbacks for parameter-varying system matrix
Consider the classical continuous-time LQR case with infinite control horizon.
One entry of the system matrix $A$ is represented through the scalar parameter $a\in[\underline{a},\overline{a}]=\mathcal{...
- 153
1
vote
0
answers
47
views
Definition of Colored Noise in terms of White Noise
For context, I was reading Optimal Control and Estimation by Stengel. In a section centered around discussing colored noise, Stengel defines a first-order difference equation $$\tag{1}x_{i+1} = ax_i + ...
- 124
0
votes
1
answer
82
views
A variational problem from physics / quantum mechanics?
I look at the problem of minimizing the functional $(a, b > 0$)
$$\int_{\mathbb R} (a x^2 f(x)^2 + b f'(x)^2)dx$$ over $L^2$ functions with ($L^2$-)norm $1$.
The solution is Gaussian but I do not ...
- 419
0
votes
0
answers
65
views
calculus of variations over the class of lipschitz function
If we have a random variable $X \sim U(0,1)$ with probability density function $f_X(x)\in[0,1]$, we might be interested in solving the following optimization problem:
$$
\max_g \ \mathbb{E} \left[ g\...
- 1
0
votes
0
answers
56
views
A question about a proof of "Optimal Control" by R. Vinter
I don't understand the proof of the Proposition 2.3.4. in the book "Optimal Control" by Richard Vinter.
Proposition 2.3.4.: Consider a function $\phi\colon I\times \mathbb{R}^{n} \times\...
1
vote
2
answers
93
views
Differential Riccati equation for infinite time horizon optimal control
Can I solve the infinite time horizon problem with the differential Riccati equation instead of the algebraic equation? As far as I know, considering the normal LTI system as $\dot{x} = Ax +Bu$ with ...
