Questions tagged [credible-interval]
A credible interval is an interval in Bayesian statistics that includes the true value of a parameter with $100(1-\alpha)\%$ posterior probability. Credible intervals treat the interval as fixed and the parameter as random.
191 questions
2
votes
0
answers
68
views
How can I best evaluate whether uncertainties (confidence/credible) are of appropriately width, in simulations?
I would like to evaluate how well two experimental designs perform with the goal of parameter estimation. I'm generating 1000 simulated datasets for each design and fitting the same model to all of ...
- 21.2k
1
vote
0
answers
39
views
How to interpret posterior uncertainty (e.g., credible intervals) under model misspecification?
In Bayesian inference, when the model is well-specified, and the prior is reasonable with respect to the true parameter of the model, the posterior is guaranteed to be well-calibrated under fairly ...
- 121
4
votes
1
answer
139
views
Loss function that is minimized by an HPD interval with specific coverage?
This answer describes two loss functions for Bayesian credible intervals, each of which is minimized by a particular kind of interval. I am curious whether there exists a loss function on credible ...
- 633
17
votes
2
answers
976
views
Shortcuts in a Bayesian power analysis without upsetting too many people
Apologies for the clickbait title - it's a reference to the fact that I know it's legitimately triggering to try to bring the frequentist concept of a power analysis into Bayesland where it doesn't ...
- 1,241
1
vote
0
answers
39
views
Making Aggregate Estimates in BAPC Package
I am working with the BAPC package in R to performing cancer incidence rates and cases projections. This is a special wrapper over the ...
- 662
1
vote
1
answer
70
views
Credible evaluation of trained model needs far more episodes than heuristic policy
During training and evaluating network model on an aircombat environment by PPO rl algorithm, it was surprisingly found that, credible evaluation of trained model needs far more episodes than ...
- 31
2
votes
0
answers
170
views
Why is the 95% confidence interval not also 95% credible?
I understand that in the frequentist's interpretation of probability, it makes no sense to say that the parameter has 95% probability to fall into this particular interval (since the parameter is not ...
- 31
1
vote
0
answers
49
views
Using Credible Interval and Mean from a previous study to Inform a Prior
I have a credible interval and mean from a previous study for a parameter and I’m wondering how to use that to inform the prior for my current study. My initial thought was to use a normal ...
- 31
1
vote
0
answers
37
views
Calculating one-sided credible intervals from posterior samples
I have a foundational question when it comes to calculating Bayesian credible intervals from posterior samples. For simplicity let's say I have a vector of 10,000 posterior samples $\theta^* = (\theta^...
- 39
5
votes
2
answers
303
views
How to interpret large intervals in GAMM mgcv::gam.vcomp() output?
I constructed a generalized additive mixed effect model (GAMM) that converged, but has very large intervals for some terms and I'm not sure if it's an issue. Practically all diagnostic tools that I've ...
- 2,537
0
votes
0
answers
53
views
What is the true meaning of Confidence level and o(x) function in ordering algorithms?
I am trying to understand the confidence level, Neymann constructions, sorting algorithms for confidence regions but I am struggling.
Tell me if I understand correctly:
The confidence level CL is the ...
- 1
3
votes
1
answer
196
views
Estimating High Confidence Upper Bound for Quantile of Unknown Distribution
I am given a random variable $X$ over $\mathbb R$ with an unknown distribution.
I want to determine the smallest sample size $n$ needed to obtain a high confidence upper bound $\hat Q_{1-\alpha}$ for ...
- 51
6
votes
3
answers
964
views
confidence intervals for proportions containing a theoretically impossible value (zero)
This is really a hypothetical question not related to an actual issue I have, so this question is just out of curiosity. I'm aware of this other related question What should I do when a confidence ...
- 93
1
vote
1
answer
67
views
Extract credible/confidence interval of a threshold in a Bayesian posterior draws distribution [closed]
I have a Bayesian model created through bayer package in R on which I need to calculate confidence/credible intervals for a ...
- 229
6
votes
2
answers
859
views
Highest-density vs equal-tailed confidence interval
When a sampling distribution is symmetric (and I'm okay assuming unimodal too, if necessary), it's natural to center confidence intervals around the point estimate. But for a skewed distribution (e.g. ...
- 53
7
votes
1
answer
473
views
Gaussian Process: confidence interval vs prediction interval vs credible interval
Let a distribution over functions be described by a Gaussian Process (GP) prior, following the notation of Rasmussen and Williams:
$$
f(\mathbf{x})\sim\mathcal{GP}(m(\mathbf{x}), k(\mathbf{x},\mathbf{...
- 73
1
vote
0
answers
199
views
Region of Practical Equivalence (ROPE)
I seek to understand how to employ the Region of Practical Equivalence (ROPE), so I conducted a basic Bayesian analysis of disease incidence rates on a public data set.
Is my rationale sound regarding ...
- 143
2
votes
1
answer
76
views
Can we report a credibility or confidence interval for a quantity measured only once, but whose distribution is obtained by Bayesian methods?
Suppose you use Bayesian methods to calculate the probability density function (pdf) of a quantity of interest $X$, given its measured value $x$ (measured only once) and some other assumptions/...
- 831
1
vote
0
answers
91
views
How to determine if there is a statistically significant difference between the predictions of two Bayesian models
I would like to determine if there is a statistically significant difference between the predictions of two Bayesian models.
Model1 predicts the distance travelled on foot during daylight based on ...
- 21
0
votes
0
answers
85
views
What's a sensible way to put a credible interval on samples in [0, 1]?
Say I take a bunch of measurements of a discrete random variable in {0, 1} (coin flips of a weighted coin for instance). I understand that a reasonable way to report on the mean and credible interval ...
- 611
1
vote
0
answers
141
views
Correct interpretation of Bayesian credible intervals
I'm starting to learn about Bayesian statistics for my own personal education (this is not a homework question). Something puzzles me a bit about the interpretation of credible intervals.
From the few ...
- 57
6
votes
1
answer
204
views
Discontiguous credible interval (take out the middle)
As far as I know, credible intervals usually involve a single contiguous interval and allow one to make statements such as: "Based on the model, data, and priors, $X$ is within the bounds $(x_1, ...
- 63
0
votes
0
answers
73
views
Highest posterior density [duplicate]
How is to find the value of $k$ for the following equation which gives $100(1-\alpha)\%$ highest posterior density?
$$\int_{\theta:\pi(\theta|\mathbf x)>k}\pi(\theta|\mathbf x)d\theta=1-\alpha,$$
...
- 369
2
votes
1
answer
240
views
Is it possible to merge credible intervals from different Bayesian prediction models into a single estimate?
The situation
Imagine an archaeological site, 10.0m deep. For my study, I construct an "age-profile" for this site, i.e., I produce a model of age as a function of depth. There are various ...
- 41
0
votes
0
answers
121
views
Credible interval for a quantile from log-normally distributed data
Both reviewers of my latest article suggested better to work with credible intervals instead confidence intervals. Unfortunately, I am not familiar with Bayesian statistics. Until today, I was (...
- 45
0
votes
0
answers
61
views
Best way to show one Bayesian model is more certain and accurate than another, based on simulated data?
I'm trying to compare performance of two bayesian models $A$ and $B$ on simulated data. It's a recruitment curve fitting problem and I'm interested in how accurate these models are in estimating only ...
- 241
3
votes
1
answer
104
views
Estimating posterior of proportion of positives in population from per-observation probabilities
I have a sample from some population of 0s and 1s and need to estimate the posterior of the proportion of 1s in this population. But the catch is: for each observation in the sample I only have ...
- 131
1
vote
1
answer
75
views
Credible interval estimation for a difference of paired binomials
Assume one is given two series of binomially distributed data:
$$X_{1k} \sim Binom(n_{1k}, p_{1k})$$
$$X_{2k} \sim Binom(n_{2k}, p_{2k})$$
$$k\in \{1\ldots K\}$$
The goal is to estimate $p_2 - p_1$ ...
- 1,048
3
votes
1
answer
231
views
Comparing three posterior distributions
I estimated a variable for three different species and have a posterior distribution of 4000 estimates for each species. Now I want to know if the distribution between species differs significantly. ...
- 321
3
votes
1
answer
912
views
Differences between HPDI and PI intervals
In Bayesian statistics, we may want to determine at what interval for example 95% of the posterior probability exists. For this we may want to use the Highest Posterior Density Interval (HPDI) which ...
- 429
2
votes
0
answers
45
views
How to compute a rectangular credible region from samples
Given high-dimensional Monte Carlo samples $\bf{X_1},...,\bf{X_N}$ from a probability distribution $p({\bf x})$ in $\mathbb{R^d}$, I want to estimate a rectangular highest-density credible region for $...
- 31
13
votes
1
answer
753
views
Why do Bayesians care about the frequentist properties of Bayesian credible intervals?
I've been doing some reading on the topic of credible vs confidence intervals but unfortunately it feels like the more I read the more I'm confused. There seems to be a general sense or consensus that ...
- 331
0
votes
0
answers
55
views
May we choose the confidence interval's width or credible interval'swidth based on a loss-function or cross-validation?
May we choose the confidence interval or credible interval based on an empirical risk loss-function?
I think we blindly generate the width intervals without any optimization or rely on some ...
user318514
1
vote
0
answers
70
views
Closed form credible region/marginal credible interval for Gaussian distribution
Suppose we have a multivariate Gaussian distribution $X \sim \mathcal{N}(\mu,\Sigma)$ where $\mu=(\mu_1,\mu_2,\dots,\mu_n) \in \mathbb{R}^n$ (the posterior of a conjugate Gaussian prior perhaps). Is ...
- 331
2
votes
1
answer
548
views
How to calculate Bayesian marginal credible interval?
I have a Bayesian model that I've fit using Stan, and I'm trying to figure out the best way to calculate the correct credible interval that I am interested in.
The model is a hierarchical GLM with a ...
- 1,087
0
votes
0
answers
131
views
How is highest posterior density interval estimated in this code snippet?
I found the following (Julia) implementation for estimating the highest posterior density interval from a posterior sample (link). Below, I turn it into pseudocode for simplicity.
...
- 43
0
votes
0
answers
163
views
Wide prediction intervals for short time series: how to fix that?
I have very small time series data(24 points) for sales for different-different regions. I need to build Range Forecast (Confidence/Credible Intervals) for sales around it for every region. I tried FB'...
- 116
0
votes
0
answers
152
views
Limit of $\frac{\lambda}{\chi _{2Y}^{2}}$ as $Y \sim \textrm{Poisson}(n\lambda)$ and $\lambda\to\infty.$
There are the following lines in Casella & Berger on page 438, before the equation (9.2.22):
..., write $$\lambda = \frac{\lambda}{\chi _{2Y}^{2}}\chi _{2Y}^{2}$$
where $\chi _{2Y}^{2}$ is a chi ...
- 29
3
votes
2
answers
231
views
How to fix hypothesis testing: MBF versus "directional correctness"
A common misinterpretation of a p-value is that it represents the probability of a false positive in the context of hypothesis testing. Here a "positive" means rejecting the null.
There are ...
- 2,170
1
vote
0
answers
72
views
What is the relationship between model uncertianty and model parameter count?
I'm looking for references, information and/or existing theory behind the relationship between the uncertainty in a given model vs its complexity/parameter count. The situation I have in mind is using ...
- 61
5
votes
0
answers
189
views
Using the terminology "Bayesian confidence interval" in place of "Bayesian credible interval."
In Peter Hoff's "A first course in Bayesian statistical methods," he states:
"Most authors refer to intervals of high probability as 'credible intervals' as opposed to confidence ...
- 73
0
votes
0
answers
2k
views
How to find 95% credible interval of a posterior predictive distribution?
I obtained a posterior predictive with 1,000 samples using MCMC, and I need to quantify the 95% credible intervals.
I know that the difference between confidence intervals and credible intervals. One ...
- 1
1
vote
0
answers
66
views
Why do we trust credibility intervals to contain the true parameter?
I understand confidence intervals and to what extent they can be trusted (and why).
However, I’m not so sure how to motivate why I should trust credibility intervals except insofar as they can also ...
user233429
4
votes
1
answer
139
views
D&D - Confidence Interval for enemy armor class
In dungeons and dragons, characters and monsters have two properties called Attack Bonus($AB \, \in \, \mathbb{Z}$) and Armor Class($AC \, \in \, \mathbb{N}$). Let $AB_c$ be the character attack bonus ...
- 1,233
1
vote
1
answer
247
views
Practical consequences of wrong interpretation of confidence intervals
Can you give me a simple (but preferably common) example (or R/Python simulation) of what are practical consequences of wrong interpretation of frequentist confidence intervals? Especially when they ...
3
votes
0
answers
849
views
mcmc vs the bootstrap
What is more accurate, the mcmc derive 95% credible interval or the bootstrap derived 95% confidence interval? Can this be proved mathematically?
the emphasis of the Bayesian approach is that one is ...
- 113
0
votes
1
answer
98
views
Statistical conclusions from incompatible measurement results
I assume that the following situation is not uncommon in scientific practice:
Two research groups analyse two samples.
The reported results concern the same physical quantity at the same location ...
1
vote
0
answers
286
views
Showing calibration of Bayesian credible intervals
I'd like to try and understand how one can prove that a particular strategy for assessing correctness of computational methods for Bayesian inference is sound.
For a number $M$ of simulations, ...
6
votes
2
answers
1k
views
When does a Bayesian 95 % credibility interval have 95% coverage?
Consider the generative model
$$
\begin{align}
\theta &\sim \pi(\cdot \mid \phi),\\
y \mid \theta &\sim f(\cdot \mid \theta).
\end{align}
$$
Compute the posterior distribution $p(\theta \mid y)...
6
votes
2
answers
926
views
Converting a confidence interval into a credible interval
The problem of correctly interpreting confidence intervals has been discussed at length here. I have a related question which I believe may be a useful contribution: Frequentist probabilities by ...
- 93