I am doing a project where I want to estimate the likelihood of an agent's mixed strategy for a current state given a list of previous state action pairs. I want to find a prior distribution on the agent's mixed strategy so that I can estimate the uncertainty on the mixed strategy. I needed some kind of prior distribution such that I can get a less variant estimation of strategy given more examples.
I define an agent's strategy as the categorical distribution $\mathbf v = \displaystyle\sum_{i=1}^{|S|}s_i\mathbf v_i$ where $|S|$ is the dimension of the state vector.
For all $\mathbf v_i$, $\mathbf v_i \sim \mathbf{Dir}(\mathbf{\alpha_i})$. In other words, each $\mathbf v_i$ has its own unique Dirichlet distribution.
How should I find the likelihood of a list of state action pairs $(x_1, \mathbf s_1), (x_2, \mathbf s_2)\dots(x_N, \mathbf s_N)$? And how can I calculate the variance on each dimension of $\mathbf v$?
I have not found anyone else implementing this strategy, and have been having a hard time computing the pdf of $\mathbf v$. I would also appreciate any suggestions for better prior distributions, but I do want to keep the properties that:
- The categorical distribution is affected by the state vector.
- We can get a better idea of what the categorical distribution for a state is given more state action pairs.
