I have a problem that I haven't encountered before and would like to know if there is any literature on the problem - or maybe you can help me simplify my problem if you think I'm doing something wrong or over-complicating things.
Assume $X_t$ is the non-Markovian stochastic process for my system's state and that $X_t \in S=\{1,2,3,4,5,...m\}$. For each time $t$, I also have a filtration $\mathcal{F}_t$ that contains everything I need to optimize my cost function, including distribution information for future stage and state costs and transition probabilities.
My policy has to be developed at the subsequence level. That is, for a given state subsequence $\{s_k\}$, I define the following increasing sequence of optional stopping times $\{\tau_i\} = \inf \{t : t>\tau_{i-1} \wedge X_{t} = s_i \}$ where I make my decisions.
For example, given a subsequence of ${s_k}=\{1, 2, 3\}$ and a sequence of states generated by my process $\{X_t\}=\{\textbf{1},4,5,4,5,\textbf{2},8,7,5,\textbf{3}\}$, the state generated these optional stopping times $\{\tau_i\}=\{1, 6, 10\}$ and we have that $\{X_1=1,X_6=2,X_{10}=3\}$ was the trajectory observed by my decision making system.
Moreover, I also have a stopping time at $T$ where the entire process is restarted. That is, continuing the example, if $T=8$, then my decisions would only be made at $\{X_1=1,X_6=2\}$ and I'd have an incomplete decision making process with the final state of my system equal to $X_{8}=7$.
Have you ever seen something like this before, and if so, where can I read more about it? If you think I'm overcomplicating things, please let me know. I am currently using Monte Carlo simulations to optimize the decision policy and was wondering if this is a problem that has already been studied significantly.
