Suppose that I have a set of observations indexed by $i$. Each observation belongs to a group $g$. Let's define by $\hat{Z}_i=Z_i - E\left[Z_i\vert g(i)\right]$ is the residual after projecting $Z$ on the groups $g$.
I have the following system to estimate: $$ Y_i=a+\beta X_i +\varepsilon_i $$ and $$ X_i=b+\alpha \hat{Z}_i + u_i $$ where I know that $E\left[\hat{Z} \varepsilon_i \right]=0$. So I can (I think) estimate $\beta$ by using $\hat{Z}$ as an instrument and where $\hat{Z}$ would be constructed as the residual after a regression of $Z_i$ on a set of groups fixed effects.
I would like to know if I could use $Z$ directly as an instrument instead if I include fixed effects in the first and second stage equations? I.e. if I write the system $$ Y_i=a^0+FE_g+\beta X_i + \varepsilon^0_i $$ and $$ X_i=b^0+FE_g+\alpha^0 Z_i + u^0_i, $$ is it true that $E\left[Z \varepsilon_i \right]=0$? Intuitively it feels like this should work but of course my intuition can be wrong and I would like to have a formal proof of this. Thank you!