1
$\begingroup$

Setup

Assume $p_Y \in \Delta^n$ is a probability vector obtained by $p_Y=L_{Y|X}p_X$, where $L_{Y|X} \in \mathbb{R}^{n \times m}$ is an arbitrary likelihood (i.e, a column stochastic matrix) and $p_X \in \Delta^m$ another probability vector.

Further consider the Bayesian posterior matrix $P_{X|Y}(u) \in \mathbb{R}^{m \times n}$ for some prior $u\in \Delta^m$, given by

$$P_{X|Y}(u) = diag(u) L_{Y|X}^T diag(L_{Y|X}u)^{-1}.$$

Entry $ij$ of this matrix contains the Bayesian posterior probability $p(x=j|y=i)$.

I'm interested in the following fixed point iteration: Starting with an initial $u_0$, repeatedly compute the posterior with prior $u_k$, averaged with respect to $p_Y$, that is

$$u_{k+1} = P_{X|Y}(u_k)p_Y = P_{X|Y}(u_k)L_{Y|X}p_X.$$

Problem

What statements can be made about the Shannon entropy of the distributions $u_k$ when the iteration is initialized with the uniform distribution $u_0 = (\frac{1}{m}, \dots, \frac{1}{m})^T$?

In particular, does it hold that $H(u_k) \geq H(p_X)$ for any $k$?

I've had some success for $u_1$ via majorization (since the matrix $P_{X|Y}(u_0)L_{Y|X}$ is doubly stochastic), but not further. What tools could be used to try to prove this or similar statements, or find a counterexample?

$\endgroup$

0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.