Questions tagged [unbiased-estimator]
Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.
792 questions
4
votes
0
answers
112
views
Unbiased estimator for 1/Var(X)
Suppose I have iid observations $(X_1, \dots, X_N)$. Is there an unbiased estimator for $1 / \text{Var}(X)$?
Clearly, we can't just take the reciprocal of an unbiased estimator for $\text{Var}(X)$; by ...
- 41
1
vote
0
answers
117
views
Is the sample MSE an unbiased estimator of any quantity?
We assume that there is a true function $f$ such that $y_i = f(x_i) + \varepsilon_i$. In any general regression setting, a common measure of the quality of the estimated true function, $\hat{f}$, is ...
- 241
1
vote
0
answers
88
views
Intuition of Bessel-like corrections for higher-order moments
Bessel's correction is understood as an adjustment to the sample variance that renders it an unbiased estimator of the population variance for an iid sample. This is generalized for higher central ...
- 573
2
votes
1
answer
117
views
Minimizing MSE for the estimator of population variance
The screenshot below is from the Wikipedia page on Mean Squared Error. Can someone please either
help me understand the last highlighted sentence (given the claim is correct),
or confirm my suspicion ...
- 137
5
votes
1
answer
150
views
Deriving unbiased estimator for standard deviation with gamma distribution
For some $X_1,...,X_n \sim N(\mu, \sigma^2)$ with both parameters unknown, I'm trying to derive the unbiased estimator for $\sigma$ from the MLE.
We know that the $\hat{\sigma} = \sqrt{\frac{1}{n}\...
- 135
2
votes
1
answer
88
views
In NLLS, how do you produce accurate estimates of RMSE(true_params) given RMSE(global_minimum_params)?
I have an exponential decay
$f(t) = \sum_n \left( A_n e^{-\frac{t}{\tau_n}} \right) + c + \epsilon(t)$,
where n represents the different exponential decay components, $A_n$ represents each decay ...
- 21
3
votes
1
answer
273
views
When is a maximum likelihood estimator biased? [duplicate]
It is known that maximum likelihood estimators (MLE) can be biased. Can we predict whether a given distribution and parameter of interest will produce a biased MLE? On what properties does it depend? ...
1
vote
1
answer
136
views
Finding the variance of an unbiased estimator [duplicate]
I tried finding the Cramer Rao Lower Bound for Variance and got (λ/n)*exp(-2λ). Then I got stuck as the UMVUE doesn't coincide with the CRLB so can't exactly find the variance of exp(-λ), I only ...
- 63
13
votes
5
answers
2k
views
What is the point of unbiased estimators if the value of true parameter is needed to determine whether the statistic is unbiased or not?
The definition that I have for an unbiased estimator is: "A statistic used to estimate a parameter is an unbiased estimator if the mean of its sampling distribution is equal to the value of the ...
- 133
2
votes
1
answer
109
views
Can the bias of an estimator depend on the number of observations? Practical interpretation of the expected value
Here it says that
$\hat{\theta}=\frac{1}{n} \sum x_i + \frac{1}{n}$
is a biased estimator of the sample mean.
Let's see:
\begin{align}
\mathbb{E}(\hat{\theta}) &= \frac{1}{n} \sum \mathbb{E}(x_i) +...
- 3,403
2
votes
1
answer
65
views
Question about Unbiasedness of Raw Sample Moments, Odd Simulation Results
I'm performing a Monte Carlo simulation to confirm that $$m_k'=\frac{1}{n}\sum_{i=1}^n X_i^k$$ is an unbiased estimator for $\text{E}[X^k]$. In the event $X\overset{iid}{\sim} \chi_1^2$, we have $\...
- 55
5
votes
2
answers
453
views
How to find a de-biased estimator with a ML component in my contaminated data problem?
I am trying to use the output of a machine learning model to estimate (using a maximum likelihood approach) a parameter in a distribution. The estimator I get has a bias which is much larger than the ...
- 166
1
vote
0
answers
39
views
Calculating population estimators when number of units sampled in a stratum is $\leq$ 1
A survey of 100 houses across 20 states in a country is conducted with each state being used as a stratum. To find the population mean I know i need to use the formula
$$\bar{y}_{str}=\sum^{20}_{h=1} \...
1
vote
0
answers
47
views
Is $\mathbb{E}\left[\|\hat{\Sigma}\|_F\right]=\|{\Sigma}\|_F$?
In one paper I read, the authors write
$$
\mathbb{E}\left[\|\tilde{\Sigma}^{-\frac{1}{2}}\left(\hat{\Theta}-\Omega\right)\|_F^2\right]=\mathbb{E}\left[\|{\Sigma}^{-\frac{1}{2}}\left(\hat{\Theta}-\...
- 209
0
votes
0
answers
64
views
Underestimation of Empirical Coefficient of Variation When Sampling from a Log-Normal Distribution with High CV
I am working with a log-normal distribution where I input a coefficient of variation (CV) to generate the variance. I then sample 𝑛 times from this distribution. The issue I am encountering is that ...
- 41
2
votes
1
answer
580
views
Efficient influence function with interventions that depend on the natural value of the exposure
Figure A1 shows a SWIG with L being a confounder of the association between exposure X and outcome ...
- 926
1
vote
0
answers
50
views
Variance of Standard Deviation in Linear Regression [closed]
Consider the linear regression setup $$y=X\beta+\epsilon$$ with $\epsilon\sim N(0,\sigma^2)$ and thus $y|X\sim N(X,\sigma^2 I)$. Let $X$ be $n\times p$ and $Y$ be $n\times 1$.
I understand that an ...
- 11
1
vote
1
answer
87
views
Regarding the definition of unbiasedness and connections to MSE
My impression is that the definition of an unbiased estimator is always made for a single parameter. Is there a reason that the definition cannot be extended for a vector of parameters? For example, ...
- 147
3
votes
1
answer
476
views
Why is "unbiased" estimator more important than min-error estimator?
According to Edwin Jaynes (Chapter 17 of his book Probability Theory: the Logic of Science), the mean squared error of an estimator consists of bias term and variance term, that is:
$$L =E[(\beta - \...
- 471
1
vote
0
answers
48
views
Mean squared error of an estimator of samples that are uniformly distributed in (-a,a)
Let $MSE(\hat{\theta}) := \mathbb{E}((\hat{\theta}-\theta)^2)$ be the mean squared error of a statistic $\hat{\theta}$.
My question is at the end of the post. The rest is my workout.
Let $X_{1},...,...
- 111
4
votes
1
answer
254
views
Statistical test for bias in a simulation study to tell if the estimate is biased or unbiased
Say we have a classic simulation study: we choose true parameter value $\theta$, then we generate N datasets and on each of those we run the model we want to test. So we get N estimates $\hat{\theta}...
- 6,327
3
votes
1
answer
141
views
How to Hyperparameter Tune without sample Bias?
While searching for ways to fine-tune the hyperparameters (HP) of my models I found out multiple reference to Cross Validation Techniques (K-folds, LPO, OOB.632+) and Ways to Select the Best ...
- 31
1
vote
0
answers
62
views
Motivation behind this exercise problem on complete sufficient statistic
This is from Hogg and McKean's "Introduction to Mathematical Statistics" Chapter 7 (Sufficiency), section 7.4 (Completeness and Uniqueness).
Exercise 7.4.10.
Let $Y_1 < Y_2 < \cdots &...
4
votes
2
answers
374
views
Deriving MSE($\hat{\beta}$) under Linear regression
I was able to derive the MSE, but there's a part of the derivation which I don't really get. Here's what I got:
Facts:
$\mathbb{E}(\hat{\beta})=\hat{\beta}\space$ (unbiased estimator)
$\text{Cov}(\...
- 391
3
votes
1
answer
83
views
Unbiased estimate of success probability
Typically we assume independence to estimate the probability of success i.e. probability of head in a coin tossing example. Means we toss a coin $n$ times and see how many times we get ...
- 195
1
vote
1
answer
214
views
Prove that $T$ is a complete statistic and find a UMVUE for $p$
While preparing for my prelims, I came across this problem:
Let $X_1, X_2,\cdots, X_n$ be a sequence of Bernoulli trials, $n \geq 4.$ It is given that, $X_1,X_2,X_3 \stackrel{\text{i.i.d.}}{\sim} Ber(\...
- 23
0
votes
0
answers
94
views
Unbiased Estimator of Nugget Effect
Question: I am trying the measure the nugget effect, which is parameterized by $(1-\lambda)$ in the following variance-covariance used to describe the multivariate normal distribution of my n-...
- 425
2
votes
0
answers
102
views
Unbiased estimator of mean divided by square root of second moment [closed]
Let $X$ be some random variable. Assume that
$$\mu = \mathbb{E}X,\,\delta = \sqrt{\mathbb{E}X^2}$$
are well defined and finite (in other words $X$ has first two moments). Now suppose that $X_1,...,X_n$...
- 121
2
votes
0
answers
401
views
What estimator and R package can be used for staggered difference-in-difference with (non-panel) cross-sectional data, controls and interactions
I am trying to run a difference-in-difference analysis in R. My data is non-panel, so I am reliant on a TWFE model where I have groups of individuals who are ...
- 65
1
vote
0
answers
72
views
Question on nonlinear least squares
Consider the following equation for $Y>0$:
$$
(1) \quad \log(Y)=\log(\gamma)+\log(\alpha+\beta X)+\epsilon.
$$
Assume that $E(\epsilon| X)=c\neq 0$. What are the consequences of this assumption on ...
- 1,016
0
votes
1
answer
269
views
Proving an Estimator of the sample variance to be MVUE
Question: Prove that $\hat{\sigma}_x^2=\displaystyle\frac{1}{N-1}\sum_{i=1}^N(X_i-\overline{X})^2$, with $\overline{X}=\frac{1}{N}\sum_{i=1}^N X_i$ is an unbiased, minimum variance estimator of the ...
- 103
1
vote
0
answers
112
views
Degrees of freedom for biased sample autocorrelation function
I want to find the expression for the a biased estimate of the autocorrelation function for a time series $X$, and am doing this from the biased estimated autocovariance function for lag $k$, divided ...
- 704
0
votes
1
answer
854
views
Why weighted importance sampling is a biased estimator?
By simple math, we can have
$$
E_P[f(X)] = \sum_X f(x)p(x) = \sum_X f(x)\frac{p(x)}{q(x)}q(x) = E_Q[f(X)\frac{P(X)}{Q(X)}],
$$
which can be approximated by Monte Carlo sampling in two ways.
1. Normal (...
6
votes
1
answer
255
views
When to calculate the bias corrected geometric mean
Most sources give a simple equation to compute the geometric mean (GeoMean) of data samples from a lognormal distribution.
GeoMean = exp(m)
where m is the mean of ...
- 22k
7
votes
1
answer
180
views
On unbiasedness of an optimal forecast
Diebold "Forecasting in Economics, Business, Finance and Beyond" (v. 1 August 2017) section 10.1 lists absolute standards for point forecasts, with the first one being unbiasedness: Optimal ...
- 71.5k
1
vote
1
answer
156
views
Unbiased estimator for mean
The question: Given a random sample $X_1,...,X_n$ show that $\frac{1}{n}\sum_{i=1}^n X_i$ is an unbiased estimator for $E(X_1)$.
My confusion: Given a statistical model $(\Omega,\Sigma,p_{\theta})$, ...
- 391
1
vote
2
answers
204
views
Covariance of Best Linear Unbiased Estimators and arbitrary LUE
I'm working on a problem involving two linear unbiased estimators $T$ and $T'$ of a parameter $\theta$, defined from a sample $\{X_1, \dots, X_n\}$ with mean $\theta$ and finite variance. I aim to ...
- 13
4
votes
1
answer
192
views
Why does not this underlying hypergeometric distribution lead to unbiased estimators?
This example is take from Lippman's "Elements of probability and statistics".
Let N be the number of fish in a lake the warden wants to estimate. He catches 100 fish, tags them and releases ...
- 307
0
votes
0
answers
113
views
Do you change the mean / standard deviation when calculating the unbiased normalised autocorrelation function?
I am trying to calculate the unbiased normalised autocorrelation function. I think this field is a little complicated as different sources appear to use different nomenclature to describe the same ...
- 206
2
votes
0
answers
60
views
How to estimate the age of players correctly?
I have the data of players active on a gaming console and the playtime hours corresponding to the games they have played and their age. I want to analyze the top (say 10) games that the people between ...
- 333
9
votes
1
answer
170
views
Adjusted R2 and bias
Consider the population $R^2$:
\begin{equation}
\rho^2 = 1- \frac{\sigma^{2}_u}{\sigma^{2}_y}
\end{equation}
This equation describes the proportion of the variation in $y$ in the population explained ...
- 285
1
vote
0
answers
82
views
What are the uniformly minimum variance unbiased estimators (UMVUE) for the minimum and maximum parameters of a PERT distribution?
I believe the answers to this question are the sample minimum and the sample maximum, but I have not been able to find a reference or proof of this.
- 11
1
vote
0
answers
67
views
Showing that the estimator of the log posterior used in stochastic gradient MCMC is unbiased
In the SGLD paper as well as in this paper it is claimed (paraphrasing) that
the following estimator:
$$\widetilde{U}(\theta) = -\dfrac{|\mathcal{S}|}{|\widetilde{\mathcal{S}}|} \sum_{{x}\in \...
- 33
0
votes
1
answer
258
views
Unbiased estimator for parameter of random variables following a uniform distribution [duplicate]
Suppose $X_i$ are i.i.d. and have density $f_\theta(x) = \frac{1}{\theta}$ if $x \in (\theta, 2\theta)$ for positive $\theta$.
$(\min_iX_i, \max_iX_i)$ is a sufficient statistic for $\theta$?
To ...
- 345
4
votes
1
answer
234
views
Unbiased estimator of $\sigma^4$
In the post [here], the user asked the question
$\{X_i\}_1^n$ is random sample from $N(\mu, \sigma^2)$ with unknown parameters. Find an unbiased estimator of $\sigma^4$.
The solution uses a property ...
- 1,656
0
votes
1
answer
211
views
Misunderstanding on the use of Popoviciu and von Szokefalvi Nagy's inequalities on the variance of a unbiased estimator
Let $X_1,\cdots,X_n$ be (discrete in my case) i.i.d. and bounded between $m$ and $M$. I'm interested in bounding the variance of an unbiased estimator:
$$\mathbb{V}\left[\frac1n\sum_{i=1}^nX_i\right]$$...
- 103
3
votes
1
answer
330
views
Is the sample mean an unbiased estimator of population mean in the presence of autocorrelation?
I've seen previous questions here that the sample mean can be considered an unbiased estimator of the population mean. e.g.1, 2.
While the examples seem to refer to independent sample points, it seems ...
- 539
0
votes
0
answers
35
views
How to prove that the MLE of a uniform distribution is biased using the formula given below? [duplicate]
I've calculated the MLE of the uniform distribution on [0,theta] as maxi{Xi} but don't know how to prove it is biased. The formula I have learned to prove it is unbiased is E(θ^)-θ=0.
Was stuck on how ...
- 1
2
votes
0
answers
91
views
Unbiased estimate of log-likelihood of Markov bridge
Note: I have cross-posted this question to MathSE.
I have the following problem I am trying to solve. I have a parametric family of "transition" distributions $p_\theta(x_{i+1}\mid x_i)$ and ...
3
votes
1
answer
178
views
Partially Endogenous Regressors
If I have a linear model
$$ Y = X_1 \beta_1 + X_2\beta_2 + e$$
where $X_1$ is endogenous to $e$ but $X_2$ is not, then simply performing OLS will yield an unbiased estimate for $\beta_2$ but not $\...
- 85