I have two machine learning models for predicting some continuous variable $y$, say $y=f_1(X_1, \theta_1)$ and $y=f_2(X_2, \theta_2)$, and these models are of the same type (ANN). $X_1$ and $X_2$ represent different sets of variables. The first model achieves performance of $R^2=0.48$ on the test dataset, while the second one got $R^2=0.22$, both of them are statistically significant. I tried to directly combine the predictors as $X=\{X_1, X_2\}$, and also use ANN to build the model $y=f(X, \theta)$, the configuration of the ANN is kept unchanged. However, the final model typically gets no higher or just a very slightly higher performance than $0.48$. Are there other typically effective ways to combine the information provided by both $X_1$ and $X_2$?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I tried also meta learning, but it did not work as well. $\endgroup$tunar– tunar2024-06-30 05:19:02 +00:00Commented Jun 30, 2024 at 5:19
-
1$\begingroup$ Did you try simple averaging of the predictions? Weighted averaging? Note that you can't expect to get much improvement over the better of your two models given the difference between the $R^2$ values. $\endgroup$jbowman– jbowman2024-06-30 17:44:25 +00:00Commented Jun 30, 2024 at 17:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Hi there: What was the logic behind the combination of the two sets of predictor variables? The answer to your question lies there. That logic must guide your selection of which predictors to combine, not simply the desire to combine your two predictor sets. You need to check closely for multicollinearity in your predictors, because it sounds like some of them are linear combinations of each other. Because of that, to expect R2 to be additive, or some function of simply throwing everything together in the same pot, as you seem to think, is not warranted in any way.
