Questions tagged [statistics]
Mathematical statistics is the study of statistics from a mathematical standpoint, using probability theory and other branches of mathematics such as linear algebra and analysis.
37,751 questions
0
votes
0
answers
38
views
Entropy of observed counts given expected probability
Suppose I have a set of symbols with expected probabilities for each, and a set of n observed sequences of these symbols, each of length m. From simply looking over the array of observed counts of X ...
- 133
0
votes
0
answers
44
views
Probability of independent trials
I have a process that predicts whether a certain event will occur. I ran 4 independent trials, and in all 4 cases the event occurred as predicted. Based on these results, what can I conclude ...
- 152
0
votes
1
answer
48
views
Inverse variance weighting for dependent observations
Is there a way to estimate the mean $\mu$ of some dependent measurements $x_1$, $x_2$, $\ldots$, $x_n$ given their covariance matrix $C\in{\Re^{n\times{}n}}$ with off-diagonal elements $\sigma_{ij}^2\...
- 5
1
vote
2
answers
143
views
Explanation of why MSE and log-likelihood of a group of bernoulli distributions are not optimised at the true probability
I have 2 random variables X and Y:
$$X_i \sim Unif(0,1)$$
$$Y_i \sim Bernoulli(X_i^{exp(\delta)})$$
I wanted to be able to test how well a set of success probabilities fit to observed set of ...
- 131
1
vote
0
answers
40
views
What is the variance of the z-coordinate of the point (x,y,z) that is randomly chosen on a unit sphere? [closed]
I understand that there is a similar question posted on the forum which talks about a unit circle and the solution can be extended to solve this question but I want to prove it by mathematically ...
- 11
0
votes
4
answers
86
views
Why is the mean of a piecewise probability density function the sum of the mean of both separate intervals?
I am doing probability density functions for Calculus 2 and came across a problem where I had to find the mean for a piecewise function. I looked up how to find the mean in this case and the equation ...
- 11
1
vote
0
answers
62
views
How do I calculate the probability of contracting an infectious disease based on the data provided [closed]
Imagine there's a dentist working in a country. In this country the incidence rate of a bloodborn infectious disease is 2.7 per 100k persons per year. And let's say, for simplification purposes, that ...
- 121
0
votes
1
answer
71
views
Bound on 'absolute' mutual information.
I have random variables $X = (X_1,...,X_n)$, and $Y = (Y_1,...,Y_m)$, where $X_i \in \{0,1\}$ and $Y_i \in [0,1]$. The entries of $Y$ are independent, but the entries of $X$ are not, although $P(X|Y) =...
- 119
0
votes
2
answers
63
views
Transfer of strong consistency
Consider a sequence of i.i.d. random variables $X_1,\, \dots,\, X_n$ whose mean is denoted as $x_0$ and variance $\sigma^2 < \infty$.
From the Strong Law of Large Numbers, the empirical mean $\bar ...
- 104
2
votes
2
answers
129
views
$N$ boxes with 2 balls in each, pick a ball randomly from a nonempty box each time, what is the expectation of 2-balled boxes after picking $N$ times?
I recently have encountered the following probability problem:
Suppose there are $N$ boxes and each box contains precisely 2 balls. Each time, we pick one ball randomly from those nonempty boxes, ...
- 833
-1
votes
1
answer
93
views
Simulating the door-switching problem [duplicate]
I have come across this problem which was apparently very famous some years ago, in which a person is placed in front of 3 doors: one of them has a stack of gold behind it, and the other two have ...
- 241
4
votes
0
answers
135
views
A maybe trivial theorem: $\mathbb{E}|f(A, B, C) - f(A, B, C') | \leq \mathbb{E} |f(A, B, C) - f(A, B', C')|$?
Let $A, B, C$ be mutually independent random variables, and let $(A', B', C')$ be an i.i.d. copy of $(A, B, C)$. Then for any function $f$. Prove that
\begin{align*}
\mathbb{E}\left[ |f(A, B, C) - f(A,...
- 41
1
vote
0
answers
45
views
What is a numerically practical and safe measure of dispersion of a data set?
I am using a moving average filter to calculate the average value of my sensor measurements. The window size is $N$. I am also using Welford's algorithm for a rolling window to calculate the variance ...
7
votes
0
answers
109
views
What is the probability distribution of farthest position in 1 dimensional random walk?
Consider a particle is doing a 1-dimensional random walk starting from the origin (probability of going $\pm 1$ direction are both $\frac{1}{2}$). Let $X_n$ be the farthest (positive) position where ...
- 350
2
votes
0
answers
61
views
Why does Shumway & Stoffer use $a_k^2 + b_k^2$ to estimate $\sigma_k^2$ instead of $(a_k^2 + b_k^2) / 2$
I have a question when reading R. H. Shumway and D. S. Stoffer's Time Series Analysis and Its Application With R Examples, 5th edition. On page 181, section 4.1, it's said that
Note that, if in (4.4),...
- 61
0
votes
0
answers
44
views
Expected value of the supremum of stochastic process(over fintie set vs over infinite set)
Let $X_t, Y_t$ be stochastic processes(we may assume the process is Gaussian if it is necessary), indexed by an arbitrary set $T \subseteq \mathbb{R}^d$.
Suppose that we have that for any finite set $...
- 2,316
1
vote
1
answer
93
views
Given $p$, solve binomial probability distribution $P(Y\ge5)$ for $n$
Given a binomial distribution where the probability of a successful outcome in a trial is 80% ($p=.8$), what is the smallest number of trials ($n$) needed to be at least 90% certain that there are at ...
- 13
0
votes
1
answer
65
views
The likelihood of a noisey coin (Bernoulli variable with observation error)
Suppose we have a coin which can be in one of two states $s \in \{0, 1\}$ where $x = P(s=1)$ is the probability of "heads". We observe $n$ independent realizations of the state of the coin, ...
- 21
1
vote
0
answers
80
views
Reference request: Formal treatment of statistics
I have started reading about Mathematical Statistics with the goal to better understand part of the foundations of Data Science. At this stage, I am particularly interested in statistical inference. ...
- 121
1
vote
1
answer
92
views
Understanding the proof of consistency of sequence of tests
I am reading about the consistency of a sequence of size $\alpha$–tests $\langle \phi_n(\mathbf X) \rangle $ where the sequence is consistent for $ \zeta\subset \Omega\setminus \omega,$ the ...
- 717
-3
votes
1
answer
53
views
Can I calculate the quantils of a distribution from the moments of the distribution?
I have a random variable $P(\vec{s}_t \vert \boldsymbol{\mu})$, where $\vec s_t, \vec s \in \lbrace 0,1 \rbrace$ are some parameters within the random experiment and $\boldsymbol \mu = (\mu_1, \ldots,...
2
votes
0
answers
51
views
Reference request for theory of estimation
I am trying to learn the theory of estimation, primarily from a mathematical (measure-theoretic/probabilistic) perspective. More specifically, I'm looking for resources that cover one-parameter and ...
- 1,353
4
votes
1
answer
306
views
Asymptotic Distribution of Weighted Empirical Distribution
Suppose that we observe an i.i.d. sample $(X_1, Y_1), ..., (X_n, Y_n)$ from $(X, Y)$. We assume that $X$ is bounded by $B$ and $E(X) = 0$. For some $\tau \in (0, 1)$, define the $\tau^{th}$ quantile ...
- 823
3
votes
3
answers
237
views
Calculating and interpreting confidence interval for a public opinion poll
Below is a problem I did. The problem is from the third edition of the book "Mathematical Statistics with Applications". The book is written
by William Mendenhall, Richard Scheaffee and ...
- 4,622
6
votes
1
answer
158
views
Sampling from two distributions to determine which is which
Peter Winkler wrote the following in his book "Mathematical Puzzles (revised edition)":
As it turns out, it’s a theorem that in trying to determine which is which of two known probability ...
- 2,231
4
votes
0
answers
72
views
A Lower Bound on Likelihood Ratio Probabilities
This refers to the proof of Lemma 11.3.2 in Hsing/Eubanks Theoretical Foundations of Functional Data Analysis.
$\newcommand{\Lo}{L_{\theta_{0}}}$
$\newcommand{\Ll}{L_{\theta_{1}}}$
We have a simple ...
- 173
5
votes
0
answers
148
views
Switching integral and derivative: a specific case
Let $X=(X_1, \ldots, X_N)$ and $Y = (Y_1, \ldots, Y_N)$ be independent zero-mean Gaussian random vector. Let $Z(t) = \sqrt{1-t} X+\sqrt{t} Y$ for $t\in [0,1]$. Let $F \colon \mathbb{R}^N \to \mathbb{R}...
- 2,316
2
votes
1
answer
84
views
Show that powers of an MVUE is an MVUE
Question is in the title. Given that $\delta:=\delta(\mathbf X_n)$ is MVUE (minimum variance unbiased estimator) of a scalar parameter $\theta$, we are asked to show that for all natural numbers $k$, $...
- 15k
0
votes
0
answers
22
views
Estimating the error when averaging a function of a matrix over a collection of random matrices
In short, I want to understand how to estimate the error in calculating the average of a function on a random matrix. I expected to be able to use the standard error of the sample mean, but that hasn'...
- 860
0
votes
0
answers
33
views
Conditional Entropy on Whole vs. Split Timeseries Dataset
I'm a novice at statistics and I don't fully grasp what I'm doing mathematically so this question isn't asking a discrete question. Only support to help intuit the math.
Suppose I have a simple time-...
- 1
2
votes
2
answers
79
views
Estimators, Bias and the Uniform Distribution
Below is a problem I did. I believe my solution is right. I am interested in feedback
about notation, style and convention. I am also wondering if my answer to part c, should
have been written as one ...
- 4,622
0
votes
1
answer
62
views
Mixtures of Gaussians' (weak) convergence property
A question about the following property: Convergence ( (weak) convergence in distribution) of mixtures of Gaussian densities to any discrete Distribution:
If we have a following GMM:
$$
f\bigl(x;\{\...
1
vote
1
answer
117
views
Why is the Hessian of the negative log-likelihood in multinomial logistic regression positive definite?
I am studying the convexity properties of the negative log-likelihood in multinomial logistic regression.
Let me briefly set up the notation:
We have a dataset
$$
D = \{(x_n, y_n)\}_{n=1}^N, \quad ...
0
votes
0
answers
55
views
Is there a way to describe the transition of two distributions?
I have a random variable $P(s)$ that approaches 1 as the sample size M is increased. $P(s)$ itself is a probability, so it is bound in $[0,1]$.
When $M=1$, the distribution of $P(s)$ is Gaussian, and ...
0
votes
0
answers
71
views
$z^{T} X(X^{T}X)^{-1}X^{T}z$ appears *exactly* Chi-squared if $z$, $X$ BOTH store Gaussian entries?
Suppose $z$ is an $N\times1$ Gaussian vector and that $X$ is an $N\times2$ ($N > 2$) matrix that contains two independent Gaussian vectors. $z$ and $X$ are independent.
Matrix $X(X^{T}X)^{-1} X^{T}$...
1
vote
1
answer
70
views
Box-Muller transformation when $Z_1=0$
When $X_1, X_2$ follow uniform distribution of $U(0,1)$
Let $$ Z_1=\sqrt{-2\ln(X_1)} \cos(2\pi X_2)$$
$$ Z_2 =\sqrt{-2\ln(X_1)} \sin(2\pi X_2)$$
Then,
$$ X_1=\exp\left(-\frac{Z_1^2+Z_2^2}{2}\right)$$
$...
- 39
1
vote
0
answers
114
views
Proof techniques to analyze a recursively defined estimator
I’m studying a recursively defined estimator and want to prove consistency. The estimator is not merely computed by a recursive algorithm—the value on a sample is defined (also) through values on sub-...
- 159
2
votes
0
answers
23
views
Quantile Regression variance attribution
For multi OLS regression. I normally use variance attribution to explain how much contribution an independent/predictor variable has to the dependent variable. For example, a standard two factor ...
- 21
1
vote
1
answer
98
views
Proof that the maximum likelihood estimator attains the Cramer-Rao bound
I want to know how to prove the maximum likelihood estimator attains the Cramer-Rao bound,
$$
n \mathrm{Var}[\hat\theta_n] \geq I_{\theta}^{-1},
$$
where $n$ is the number of repeated i.i.d. ...
- 910
1
vote
1
answer
154
views
Is there a name for the operator $⊞$ defined as $a ⊞ b = \sqrt{a^2 + b^2}$?
It's not uncommon to have a meaningful relationship c² = a² + b². This happens in both geometry (Pythagorean theorem) and statistics (sources of variance) at least, and I think I've seen it elsewhere. ...
- 193
0
votes
1
answer
60
views
For each spin, there is $1/5$ chance you win a lucky scratch; for each lucky scratch there is $1/10$ chance you win a spin. What is $E(X)$ and $E(Y)?$
In an online game, there are two types of bonuses, free spins and lucky scratches.
A "free spin" spins a wheel and you win whatever prize the spinner lands on.
A "lucky scratch" is ...
- 25.2k
-1
votes
1
answer
43
views
Using the normal distribution to approximate a binomial distribution
Below is a problem I did. I believe the answer I got is correct. It differs from the answer in the book. I believe that difference is round-off error but I am not sure. Is it round-off error?
Problem:
...
- 4,622
1
vote
1
answer
32
views
Connectedness of LRT Poisson Confidence Sets, i.e., Monotonicity of the LRT Poisson Acceptance Regions
Background: Working on Exercise 9.23 (a) in "Statistical Inference, 2nd Edition" by Casella and Berger. Simply speaking, the problem asks for a $1 - \alpha$ confidence interval for a Poisson ...
- 49
0
votes
0
answers
27
views
Add independent variable whilst performing statistical regression SSE decreases
Say we perform regression with independent variable $x_1$ and dependent variable $y$, get a best fit line and compute the SSE. Then we realize that we have data for a second independent variable $x_2$ ...
- 363
0
votes
0
answers
83
views
What do we really mean by independence of jointly distributed random variables?
I saw two questions essentially the same as mine, but neither answered my question directly, so here goes.
Consider the simple random experiment of tossing a coin, where our interest
lies in the face ...
- 433
1
vote
1
answer
34
views
Showing $\frac{\mathbb{P}\{\text{data}_t|\text{wrong pmf }\} }{\mathbb{P}\{\text{data}_t|\text{correct pmf }\}}\overset{\text{a.s}}{\to}0$.
Setup: Let $n\in\{2,3,...\}$. For each $t\in\{1,2,...\}$, suppose $X_t$ is distributed i.i.d., taking value $v\in
\{1,...,n\}$ with probability $p_v\in(0,1)$. Let $S_{vt}:=\sum_{\tau=1}^t 𝟙\{X_t=v\}$...
0
votes
1
answer
45
views
VC dimension of a left-sided interval
I'm asking a question that has been answered here as I do not understand the answer.
The question is the following: Consider the class of left-sided half intervals in $\mathbb{R}^d$:
$$\mathcal{S}^d:=\...
- 2,316
2
votes
1
answer
84
views
How many samples until the percentile estimate stops wobbling?
I am not a mathematician, just a curious computer science student.
I came up with the following problem while thinking about sampling, and I’m sure there must be related questions out there,
but I ...
- 63
1
vote
1
answer
299
views
VC dimension of monotone Boolean conjunctions
The following is an exercise from High-dimensional statistics by Wainwright.
Consider the class of functions $\mathcal{B}_d = \{h_S \colon \{0,1\}^d \to \{0,1\} \mid S \subseteq \{1, \ldots, d\}\} \...
- 2,316
0
votes
1
answer
43
views
What is the relationship between $E[\#{i: T_i \leq t, V_i \leq v}]$ and $E[\#{i: t \leq T_i \leq t + dt, v \leq V_i \leq v + dv}]$? [closed]
Setup
Let $\{N_t\}_{t\geq 0}$ be poisson process with parameter $\lambda$, i.e. $N_t \sim Po(t\lambda)$. Let $T_i$ be the time when the $i$-th accident occurs. Furthermore, let $V_i$ be the time ...
- 682