I am a CS undergrad with some basic stats courses under my belt. I am now taking a time series course, which thus far seems to just apply the same statistical concepts to time-related data, which leads to some challenges and things to consider.
I am now trying to understand what the conditions are needed for meaningfully compare correlation coefficients among different pairs of random variables. The question arose when I saw that for a random walk, the ACF is higher the further in time you are (for the same lag), e.g. rho(x_100, x_90) > rho(x_20,x_10), since the former pair share a larger path of the same white noise shocks. However, I struggled with interpreting this as higher linear predictive power, since in both cases we still observe the same amount (10) of ADDITIONAL random shocks.
This led me down a rabbit hole of trying to understand when the correlation coefficient as a measure of linear predictive power is directly comparable between pairs of random variables. Since time series are just an application of this where the random variables are connected to the same process in time, I wanted to understand this in the more general context.
I would like to know 1) what are the conditions needed to directly compare correlation coefficients between pairs and conclude "rho(X,Y) is higher than rho(A,B), therefore X linearly predicts Y better than A predicts B"
and 2) what part of the weakly stationary conditions are sufficient for this comparison to work in the time series context.
My current understanding is:
by a prediction error formula
Var(e) = (1-r^2) * Var(Y), suggesting only Var(Y) directly affects the error prediction.for each individual pair of random variables X and Y, r(X,Y) measures how well X linearly predicts Y, or the direction and strength of their linear relationship. Linear predictions implies a choice of independent and dependent/predicted random variable. The linear regression is scale-invariant for the independent variable (adjusts the slope), hence the error prediction formula will only be affected by the inherent Var(Y).
^I am not too sure of the detailed intuition, but let's just say the math checks out. I read something about this meaning relative prediction, i.e. relative to total variance, a large part is explained well by the model. In absolute units, however, larger Var(Y) will mean larger deviation in predictions. So in the example of the random walks, the (t_100 t_90) pair had higher correlation, meaning higher "relative predictive power". That is, most of the variance in T_100 is already explained well by the variance in T_90, so the linear prediction model captures most of the total variance proportion already. But this says nothing about how large in absolute units this uncaptured small proportion is.
thus, I conclude that to directly compare correlation values to see which pair can better linearly predict each other, the variance of the predicted variable must equal. Otherwise, we don't have the same absolute units to compare with.
in the context of time series, I read that weak stationarity can be assumed for this to work. Is it true then that it is only the property of constant variance that is truly needed, and that the constant mean and covariance only being lag-dependent not really relevant here?
Thank you.
