I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate.
Let $X_t$ be the state of the Markov chain at time $t$ and suppose that I can only observe the states of my Markov chain at time $\tau_1<\tau_2<\tau_3,\dots$, where $\tau_i$ is a sequence of observation times that I do not control. Is there a way to estimate the original probability matrix $P$ (if possible with some idea on how close is my estimate from the true $P$), and also to estimate $P^k$ for $k>0$.
Possible approach
If $\tau_i=i$ (i.e., if I can observe at all state), there are different method that I can use. One is to use a Bayesian approach: I consider a uniform prior on $P$ (which corresponds to a Dirichlet distribution on each line) and each time I observe a transition from $i$ to $j$, I update the $i$th line.
Now, if $\tau_{i+1}-\tau_i$ is $1$ if $i$ is even and $2$ when $i$ is odd, I can still define a posterior update on $P$ -- when I observe that $i$ jumps to $j$ in two steps -- but it is not clear to me if there is a computable way to do so. I can estimate $P$ and $P^2$ separately but I am not sure how to use my knowledge on $P$ to $P^2$ and vice-versa.
