2
$\begingroup$

Let's say that the analytical infrastructure at my place of work heavily centers around t-tests. I'd like to use this platform to perform a t-test on correlated data, e.g. the extent to which an order placed by a customer was 'satisfactory' or 'unsatisfactory' where the treatment variable is at the level of the customer and the continuous dependent variable - 'order is satisfactory' [0,1.0] - is at the order level and is clustered on customers. Our systems are set up to perform an independent two-sample t-test, i.e. $$ t = \displaystyle\frac{\bar{X_1}-\bar{X_2}}{S_p\sqrt{\frac{2}{n}}} \text{ where } S_p=\sqrt{\frac{s^2_{x_1} + s^2_{x_2}}{2}} $$ ...but my understanding is that given the clustered nature of the data, $s^2_{x_1}$ and $s^2_{x_2}$ are no longer unbiased, and more likely than not, will underestimate the true variance.

I've been advised that I can use the Delta Method to approximate the true variance, thus enabling a t-test where the standard errors are approximately correct. I've tried to go over the Delta Method, but I've struggled to understand how I can apply the insight that $$ {\displaystyle {{\sqrt {n}}[g(X_{n})-g(\theta )]\,{\xrightarrow {D}}\,{\mathcal {N}}(0,\sigma ^{2}\cdot [g'(\theta )]^{2})}} $$ .. to the estimation of $s^2_{x_1}$ and $s^2_{x_2}$.

I'm definitely a bit out of my league here, so any help (e.g. example walkthroughs of the Delta Method, reading material appropriate for beginners) would be greatly appreciated.

Thanks!

Improve this question
$\endgroup$
6
  • $\begingroup$ It's not immediately obvious why the delta method, which helps us get asymptotic distributions for transformations of variables, helps us in your case. Since the treatment is at the customer level, why not aggregate customer data by averaging, taking into account the heteroscedasticity that this implies, and perform an independent t-test at that point? PS, since your data are bounded [0,1] (and I'll take a guess and say there's probably a lot of data on the boundaries), normality of the data is a strange assumption; why use a t-test rather than a z-test? $\endgroup$ Commented Sep 9, 2022 at 17:57
  • $\begingroup$ Thank you for the reply John. I appreciate that there are alternative approaches that are likely viable (i.e. changing the unit of analysis). However, I know that the Delta Method has been used in contexts similar to what I've described above, and my motivation centers more around the 'how' versus simply getting the correct answer as quickly as possible. To your second point, I am happy to use either a t-test or a z-test so long as $s$ can be estimated appropriately. $\endgroup$ Commented Sep 9, 2022 at 19:43
  • $\begingroup$ I am also unclear as to how the delta method would help you adjust standard errors for correlation of observations. $\endgroup$ Commented Sep 10, 2022 at 1:39
  • $\begingroup$ Thanks I think I may have misinterpreted the advice. I suspect that I'm not supposed to use the Delta Method to approximate $s$. Rather, I'm supposed to used the Delta Method to approximate the standard error of the ratio of means. Specifically, I'm being told that the standard error of the ratio of means can be approximated as $$SE(\bar{Y}/\bar{X}) = (1/\bar{X}) SE(\bar{Y}-\lambda\bar{X}) \text{ where } \lambda = \frac{\mu_Y}{\mu_Y}$$ How one is supposed to use the Delta Method to arrive at this insight is beyond me. $\endgroup$ Commented Sep 10, 2022 at 17:17
  • 1
    $\begingroup$ Bit late, but this paper from Deng et al. (in particular Section 3) provides a guide on the delta method from an applied perspective and may be useful. $\endgroup$ Commented Sep 14, 2022 at 13:23

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.