1
$\begingroup$

In lavaan, I am running a two-factor CFA on a questionnaire with 28 items, all of which are scored on a 6-point Likert scale. In total I have ~350 participants who completed the questionnaire.

Because of the ordinal nature of the data, I am using ULS instead of ML, and the ordered = T command.

Because I have found outliers in my dataset (both in terms of Mahalonobis distance and generalized Cook's distance, but not in terms of standardized residuals), I want to use a robustification method to reduce the influence of these extreme cases, instead of discarding these observations, as recommended by Flora et al.

Now, based on my understanding, I can do that by choosing the estimator ULSM, ULSMV, or ULSMVS. According to the lavaan documentation, the difference is the following:

  • ULSM estimator uses "robust standard errors and a Satorra-Bentler scaled test statistic";
  • ULSMV estimator uses " robust standard errors and a mean- and variance adjusted test statistic (using a scale-shifted approach)";
  • ULSMVS estimator uses "robust standard errors and a mean- and variance adjusted test statistic (aka the Satterthwaite approach)"

Which estimator I use has a very strong effect on my CFI and RMSEA scores (but not SRMR or WRMR), as can be seen in this figure: enter image description here

However, I cannot find out which of these estimators, the ULSM, the ULSMV, or the ULSMVS, I should use. So,

  1. Am I approaching this analysis generally correct, and
  2. Which of these estimators should I use?

Thanks in advance!

Improve this question
$\endgroup$
1
  • $\begingroup$ Note: my data are not normally distributed. $\endgroup$ Commented Mar 6, 2023 at 9:38

1 Answer 1

Reset to default
2
$\begingroup$

Because I have found outliers in my dataset (both in terms of Mahalonobis distance and generalized Cook's distance

These were calculated by treating your 6-point Likert scale as a continuum, right? So 1 means 1, 2 means 2, ...

Because of the ordinal nature of the data, I am using ULS instead of ML, and the ordered = T command

The ordered = TRUE argument tells lavaan() you don't want to treat the Likert scale as a continuum, but as 6 ordinal categories.

I want to use a robustification method to reduce the influence of these extreme cases ... by choosing the estimator ULSM, ULSMV, or ULSMVS

You don't get anything robust to outliers or nonnormality. The regsem package provides robust ML to reduce the influence of outliers. If you treated your data as coming from a continuum (do not set ordered=TRUE), then you those lavaan(estimator=) options provide test statistics robust to nonnormality. When you invoke the latent-response assumption with ordered=TRUE, there is no way to adjust for the degree to which those latent responses are not normally distributed; you simply have to assume normality underlies those discrete responses.

Robitzsch, A. (2020). Why ordinal variables can (almost) always be treated as continuous variables: Clarifying assumptions of robust continuous and ordinal factor analysis estimation methods. Frontiers in Education, 5(589965). https://doi.org/10.3389/feduc.2020.589965

Which of these estimators should I use?

The *MV adjustment is generally the most robust, especially when treating data as ordinal. But your sample is so small for such a large model, I expect even your robust test statistic would not have nominal Type I error rates.

but not SRMR

That's because RMR, SRMR, and CRMR are based on residuals, not on $\chi^2$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks a lot for your elaborate answer! "These were calculated by treating your 6-point Likert scale as a continuum, right?" I think so, yes. I used the faoutlier package, and the obs.resid(), robustMD(), and gCD() functions with their default settings. "The regsem package provides robust ML to reduce the influence of outliers." Awesome! But when I look in the documentation of the package, I can't figure out how to call the robust ML... Also, is it possible to combine the robust ML with the MV adjustment, to also protect against nonnormality? $\endgroup$ Commented Mar 8, 2023 at 6:51
  • $\begingroup$ Ah, based on your comment here I am guessing that the regsem package automatically reduces the influence of outliers, and that it is indeed possible to use the MLR or MLMV estimator. Thanks! $\endgroup$ Commented Mar 8, 2023 at 7:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.