Questions tagged [model-averaging]
The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.
95 questions
4
votes
1
answer
114
views
Smooth AIC selection
Suppose I have a family of $N$ models for the same data, indexed by $n\in\{1,\dots,N\}$.
And suppose that model $n\in\{1,\dots,N\}$ has log-likelihood given by:
$$L(X_n \theta_n),$$
where $L:\mathbb{R}...
- 565
1
vote
1
answer
128
views
Misunderstanding in Bayesian model averaging
I'm making my first attempt at understanding and implementing Bayesian model averaging, to make a weighted mix of several competing models. Many of the sources that I am reading make a similar claim ...
- 13
2
votes
1
answer
161
views
GLMM Model Averaging with Predictor Multicollinearity
I am running GLMM models to determine how environmental factors influence bird collisions. I've obtained a list of candidate models with delta AIC less than 2, and I want to perform model averaging.
I ...
- 21
1
vote
0
answers
53
views
How to calculate $P(f_1(X) = \text{max}(f_1(X), \dots, f_K(X))$ when $X$ is multivariate Normal?
Let's say I have a multivariate distribution $\mathbf{X} \sim \text{MVN}(\mathbf{\mu}, \mathbf{\Sigma})$ and a set of $K$ scalar functions of $\mathbf{X}$, $f_1(\mathbf{X}), \dots, f_K(\mathbf{X})$. ...
- 40.2k
0
votes
0
answers
70
views
Computing Bayesian model averaged posteriors
The Bayesian model averaged posterior predictive distribution for new data $\tilde{y}$ given training data $y$, across a set of $M$ models $\mathcal{D} = \{D_{1}, ..., D_{M}\}$, is defined as:
\begin{...
- 185
0
votes
0
answers
61
views
Interpreting Contradictory Results in Bayesian Model Averaging: High Posterior Inclusion Probability with Unclear Effect
In my research, I am utilizing the Bayesian Model Averaging (BMA) methodology to identify the best set of regressors that can predict the outcome variable $y$. My dataset consists of five variables ...
- 37
1
vote
0
answers
126
views
Cross validation with GLMMs; best way to partition train and test data with regard to random effects?
I have some analysis I'm working on and I'm having a hard time nailing down the correct approach to take. I am modeling the dynamics of frog choruses by looking at what predicts the outcome of calling ...
- 21
2
votes
0
answers
104
views
Bayesian model averaging: trying to calculate the posterior probability
I am new to statistical modelling and I need some help with Bayesian model averaging.
I have 3 models and I would like to derive a BMA of these models.
I am using the BIC estimate for the different ...
- 21
1
vote
0
answers
38
views
General question on model weighting and averaging
I've had a stats related question I've been wondering for a while, but for which I have yet to find good sources on. I'm aware of things like the Akaike Information Criterion for weighting candidate ...
- 11
1
vote
1
answer
217
views
Weird AIC weights in model averaging
I conducted AIC selection on a set of models, and isolated my top 3 models. I then calculated the AIC weights of each model, and got the values 0.99, 1.92e-14,and 6.9e-18. I never saw weight values ...
- 161
1
vote
0
answers
54
views
Is it possible to average models with different link functions?
While I can compare models that have different link functions in terms of AIC/BIC weights, I think it's impossible to use those weights to create an averaged model. Am I right in believing this?
- 1,541
5
votes
1
answer
298
views
Bayesian model averaging
In which situation should one refrain from using BMA? It seems to me that it is always a good idea to use the posterior probabilities when inferring/predicting.
- 53
1
vote
1
answer
663
views
When/what to standardize for model-averaging with(out) interactions
I’m using the {MuMIn} package in R to select models (dredge, get top models, average etc). My question is about whether I need to, or should, standardise my ...
- 11
10
votes
1
answer
985
views
Averaging SVM and GLM results: sensible or stupid?
I have taken two different approaches to calculate probability: using a GLM and an SVM. They are giving slightly different results (which is understandable, they are completely different approaches). ...
1
vote
0
answers
74
views
BMA and RJMCMC predictive performance
We have a family of statistical models, with parameter spaces of different dimensions, which we aggregated through standard Bayesian Model Averaging (BMA). Experiments using training and test sets ...
- 25.5k
2
votes
1
answer
395
views
Does bagging work for OLS to improve prediction?
In Elements of Statistical Learning, section 8.7, the author states that
The bagged estimate will differ from original estimate only when the latter is a nonlinear or adaptive function of the data
I ...
- 343
13
votes
4
answers
2k
views
In what applications do we prefer Model Selection over Model Averaging?
I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
- 3,090
6
votes
1
answer
390
views
Is post-variable-selection multimodel inference a bad idea?
If I understood correctly, in this answer, Ben Bolker says that using inferential methods after having performed AIC-based model selection is wrong because "standard inferential methods assume ...
- 661
6
votes
2
answers
1k
views
Do "conditional" averaged coefficients *ever* make sense?
This question is related to one on Stack Overflow where the user wanted to obtain predictions and their standard errors using the "conditional" averaged coefficients from the R package MuMIn....
- 22.2k
2
votes
0
answers
472
views
Is model averaging possible/appropriate with a GAM?
I'm new here - apologies if there are any problems.
I have two GAMs that I've fit (using mgcv::gam()), and the two models are comparable in AICc values. With GLMs, ...
- 21
3
votes
1
answer
534
views
Can Bayesian Model Averaging be Optimal when the Hypothesis Space does not contain the true hypothesis?
I am utterly confused. I have been reading about the optimality of Bayes classifier and Bayesian model averaging all the time, but when I try to dig deeper, I just get more confused.
On the one hand,
...
- 798
1
vote
3
answers
163
views
How can we assume the models are exhaustive in Bayesian Model Averaging?
Bayesian model averaging is justified using the law of total probability which requires the the set of models that we average over to be exhaustive. Shouldn’t we prove that the set of models are ...
user297721
1
vote
0
answers
169
views
Model averaging parameter estimates for glmer models using AICmodavg in R
I encounter the following error trying to calculate model-averaged parameter estimates across a set of GLMM models using the AICmodavg package version 2.3.1. in RStudio(version1.2.5)
ERROR:
Error in ...
1
vote
0
answers
426
views
When is it acceptable to compute (conditional) subset-averaged coefficients?
I'm running an ecological study and I have 4 dependent variables (DVs) that I would like to explain (my interest thus lies in inference and not in prediction). For each one of these variables, I built ...
- 661
0
votes
0
answers
240
views
Model averaging a GLM function with equal weights
Background
Let me first say that I read this post and that I looked at the BMS vignette.
I used the package sure (CRAN, R ...
- 538
2
votes
0
answers
126
views
Bayesian Model Averaging (Bayesian Averaging of Classical Estimates) Issue
I am trying to implement a method used by this paper (Described briefly at the bottom of page 3):
From what I understand, it does 2$^k$ OLS regressions each time step to forecast the next time step ...
- 21
2
votes
1
answer
112
views
Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]
The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
9
votes
1
answer
724
views
AIC model averaging when models are correlated
AIC model-averaging: In "standard" AIC model averaging we average models with weights proportional to
$$w_i \propto \exp( -0.5 \times \Delta \text{AIC}_i ),$$
where $\Delta \text{AIC}_i$ is ...
- 38k
1
vote
0
answers
89
views
Using AIC weights to determine prediction intervals for a single model structure
I am working with fitting regression models to data, and producing prediction intervals would be useful. Unfortunately, the data often has few data points, and is reported as mean rather than ...
- 41
3
votes
1
answer
3k
views
Model averaging: predictor significance vs importance
What is the correct way to report the results of model averaging? I’m using MuMin in R, results include a p-value. It’s also possible to get the importance of each predictor. Example results below.
...
- 125
1
vote
0
answers
88
views
Bayesian model averaging when none of the models is well specified
Usually, from what I've read of Bayesian Model Averaging (BMA), a typical assumption is that one of the models is well specified...
However, what will happen when all models are misspecified? What ...
- 5,982
2
votes
0
answers
207
views
Why is pseudo-Bayesian Model Averaging using WAIC giving counterintuitive results for my model? Issue with unconstrained data uncertainty?
I'm currently working on a model which fits a pair of curves which are parameterized by some shape and scaling parameters. I want to produce weights for two different models. In my first model, which ...
- 21
5
votes
1
answer
356
views
Regularization via model averaging?
Say you have the model
$$ \Phi^{-1} \left(y\right)=\beta_0 + \beta_1 x$$
I am interested in adding some regularization, specifically concerning the parameter $\beta_1$, to introduce some "skepticism" ...
- 3,315
1
vote
0
answers
453
views
Is It Valid To Calculate Model-Averaged Confidence Intervals In the Same Way As Model Averaged Predictions?
I'm fitting a series of mixed-effects models, and I'm trying to calculate the model-averaged predictions and their confidence intervals. If I have a set of $R$ models $\{M_1,...,M_R\}$ I know that ...
- 113
0
votes
0
answers
277
views
When averaging models, which models need to meet assumptions?
I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
1
vote
1
answer
840
views
Is there a difference between averaging individual regressions and including a random effect?
I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different ...
- 317
1
vote
0
answers
195
views
Combining regression models from separate data sets
What is the best way to combine regression betas from separate data sets?
For example, a data set is split in two based on some fundamental characteristic, and the same two factor regression is run ...
- 11
2
votes
2
answers
2k
views
Weighting of multiple linear regressions in an ensemble
If I have a continuous dependent variable and N continuous predictors, and I fit all possible regressions with zero up to N variables, how should I weight those regressions for prediction? One ...
- 606
1
vote
0
answers
131
views
Is this Bayesian model averaging?
A classical example of Bayesian model averaging (BMA) is the regression setup where the choice of different sets of covariates corresponds to different models $\mathcal{M}_k$, $k = 1, \ldots, K$, ...
- 425
1
vote
0
answers
345
views
Can model averaging be applied on models fitted to different data sets?
I have two data sets collected from two different sets of participants on their behaviour.
EDIT: Both have the same response variables (Propensity to behave in a certain way - Yes or No). But they ...
- 11
1
vote
1
answer
260
views
BMA - at what posterior effect probability can we say that a variable has an effect?
in the setting of a Bayesian model averaging, we are not dealing with P--value when we are assesing variable importance, but with the posterior effect probability of each variable. My question is, at ...
- 125
0
votes
1
answer
236
views
Interpretation of a classic multinomial logit vs. BMA of multinomial logit
EDIT #1
Most likely I have set up the function bic.mlogit in a wrong way. @Jesper Hybel, hopefully, directed me in the right way. With the new setup I get two sets ...
- 125
0
votes
1
answer
130
views
vector of weights
I am using the MuMIn package for model averaging. However, I am not clear of the function par.avg(). In this function, we need to specify the following,
...
- 81
3
votes
1
answer
2k
views
Effect sizes for model averaging
The goal is to find if one factor is stronger than the other in the models I have considered. I am using the information-theoretic approach. Since $n/K>40$, I am using AIC. Firstly the model is ...
- 81
15
votes
1
answer
2k
views
For model-averaging a GLM, do we average the predictions on the link or response scale?
To compute the model-averaged predictions on the response scale of a GLM, which is "correct" and why?
Compute the model averaged prediction on the link scale and then back-transform to the response ...
- 656
1
vote
0
answers
253
views
Model Averaging errors using lme4 in R
I am running a glmer model on a response variable with binomial distribution and random term. My data has 3 explanatory categorical variables and I have successfully run dredge() on them and their ...
- 11
4
votes
0
answers
2k
views
Model averaging in mixed models
I am reading this blog post regarding using model averaging for mixed models and the last para says
"https://sites.google.com/site/rforfishandwildlifegrads/home/mumin_usage_examples"
...
- 991
0
votes
1
answer
392
views
Model-averaged for a glm.nb [closed]
How can i do a model averaged for a negative binomial model. I try with the AICcmodavg but it is not compatible with glm.nb models.
Any help would be appreciated
Thank you very much
Magdalena
1
vote
1
answer
624
views
How does model averaging with a categorical variable work?
I have a series of models (~14) which do not include a categorical variable. One model (#15) however, does have a categorical variable with 3 levels. Normally for this model one of the categories ...
- 21
3
votes
1
answer
1k
views
McFadden Pseudo R² in averaged model
McFadden's pseudo-R² is a well-known coefficient of determination. If I am right, it can be calculated by 1-(mod_deviance/null_deviance), where ...
- 457