Questions tagged [approximation]
Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.
72 questions
26
votes
3
answers
9k
views
Evaluate definite interval of normal distribution
I know that an easy to handle formula for the CDF of a normal distribution is somewhat missing, due to the complicated error function in it.
However, I wonder if there is a a nice formula for $N(c_{-}...
- 14.1k
58
votes
4
answers
37k
views
Approximate order statistics for normal random variables
Are there well known formulas for the order statistics of certain random distributions? Particularly the first and last order statistics of a normal
random variable, but a more general answer would ...
- 3,872
15
votes
1
answer
4k
views
Constructing a continuous distribution to match $m$ moments
Suppose I have a large sample drawn from a continuous distribution, size $n$, and $2 < m\ll n$ moments from that sample. Alternatively, suppose I have been given those moments by an angel, ...
- 3,297
32
votes
3
answers
10k
views
Difference of two i.i.d. lognormal random variables
Let $X_1$ and $X_2$ be 2 i.i.d. r.v.'s where $\log(X_1),\log(X_2) \sim N(\mu,\sigma)$. I'd like to know the distribution for $X_1 - X_2$.
The best I can do is to take the Taylor series of both and ...
- 421
3
votes
2
answers
2k
views
Natural log approximation
I've got an equation that contains
$$x^p - 1$$
$x$ is any positive number (such as 2) and $p$ is a small positive number close to 0 (such as 0.001).
For some reason (that I may have known in High ...
- 142
12
votes
1
answer
719
views
Should degrees of freedom corrections be used for inference on GLM parameters?
This question is inspired by Martijn's answer here.
Suppose we fit a GLM for a one parameter family like a binomial or Poisson model and that it is a full likelihood procedure (as opposed to say, ...
- 67.5k
10
votes
2
answers
5k
views
Variance of Normal Order Statistics
Suppose we have $X_1, \cdots, X_n \overset{\textrm{i.i.d.}}{\sim} \mathcal{N}(0, 1)$ with $n > 50$, and let $X_{(1)}, \cdots, X_{(n)}$ be the associated order statistics.
Are there any references ...
- 1,422
16
votes
2
answers
16k
views
What is the normal approximation of the multinomial distribution?
If there are multiple possible approximations, I'm looking for the most basic one.
- 345
14
votes
4
answers
6k
views
What is the CDF of the sum of weighted Bernoulli random variables?
Let's say we have a random variable $Y$ defined as the sum of $N$ Bernoulli variables $X_i$, each with a different, success probability $p_i$ and a different (fixed) weight $w_i$. The weights are ...
- 181
14
votes
3
answers
5k
views
How to compute the probability associated with absurdly large Z-scores?
Software packages for network motif detection can return enormously high Z-scores (the highest I've seen is 600,000+, but Z-scores of more than 100 are quite common). I plan to show that these Z-...
- 7,771
11
votes
4
answers
7k
views
Does the universal approximation theorem for neural networks hold for any activation function?
Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions?
Update: As shimao points out in ...
- 12.2k
6
votes
1
answer
1k
views
Bound for weighted sum of Poisson random variables
Suppose I have some independent Poisson-distributed random variables $X_1 \ldots X_N$ with parameters $\lambda_1 \ldots \lambda_N$. These can be thought of as processes where each arrival/event ...
- 203
17
votes
3
answers
42k
views
Normal approximation to the Poisson distribution
Here in Wikipedia it says:
For sufficiently large values of $λ$, (say $λ>1000$), the normal distribution with mean $λ$ and variance $λ$ (standard deviation $\sqrt{\lambda}$), is an excellent ...
- 431
15
votes
4
answers
8k
views
Confidence interval from R's prop.test() differs from hand calculation and result from SAS
I'm wondering if anyone has insight into how prop.test() in R calculates its confidence intervals. Although it doesn't state it explicitly in its documentation, my ...
- 1,923
12
votes
1
answer
1k
views
Approximate distribution of product of N normal i.i.d.? Special case μ≈0
Given
$N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$,
and $\mu_X \approx 0$,
looking for:
accurate closed form distribution approximation of
$Y_N=\prod\limits_{1}^{N}{X_n}$
asymptotic (...
- 241
25
votes
1
answer
9k
views
How does a random kitchen sink work?
Last year at NIPS 2017 Ali Rahimi and Ben Recht won the test of time award for their paper "Random Features for Large-Scale Kernel Machines" where they introduced random features, later codified as ...
- 3,136
15
votes
5
answers
25k
views
normal approximation to the binomial distribution: why np>5?
Nearly every text book which discusses the normal approximation to the binomial distribution mentions the rule of thumb that the approximation can be used if $np\geq5$ and $n(1-p)\geq 5$. Some books ...
- 721
14
votes
1
answer
3k
views
One sided Chebyshev inequality for higher moment
Is there an analogue to the higher moment Chebyshev's inequalities in the one sided case?
The Chebyshev-Cantelli inequality only seem to work for the variance, whereas Chebyshevs' inequality can ...
- 481
8
votes
3
answers
5k
views
regression with constraints
I have some domain knowledge I want to use in a regression problem.
Problem statement
The dependent variable $y$ is continuous.
The independent variables are $x_1$ and $x_2$.
Variable $x_1$ is ...
- 1,613
7
votes
3
answers
567
views
Probability for finding a double-as-likely event
Repeating an experiment with $n$ possible outcomes $t$ times independently, where all but one outcomes have probability $\frac{1}{n+1}$ and the other outcome has the double probability $\frac{2}{n+1}$,...
- 210
6
votes
1
answer
968
views
Half-normal probability plot
To construct the half-normal probability plot, plot the absolute values in a certain statistical diagnostic (residual, leverage, Cook distance and others) versus $z_i$ where:
$\displaystyle z_{i} = \...
- 99
4
votes
1
answer
161
views
Formulas or approximations for $\mathbb{E}\left( \frac{X}{\|X\|} \right)$, $X\sim N(\mu, Id)$?
This is a cross-posting of this math SE question.
I want to compute or approximate the following expected value with some analytic expression:
$\mathbb{E}\left( \frac{X}{||X||} \right)$
, where $X \in ...
- 2,362
4
votes
1
answer
427
views
In exactly what sense do MCMC draws approximate the target?
Background
We want to sample from some intractable density $\pi(\theta)$. Using an MCMC algorithm, we generate a sample of draws $\{\theta_i\}_{i=1}^N$ from a Markov chain that has $\pi(\theta)$ as ...
- 1,435
3
votes
2
answers
2k
views
Function Approximation vs. Regression
Some background before I state the questions:
I have a $d$-dimensional random vector $X=(X_1,\ldots,X_n)$ and a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$. Ultimately my goal is to understand $f$ ...
- 2,954
2
votes
1
answer
829
views
How to show that normal distribution is a second order approximation to any distribution around the mode?
How can I show that normal distribution is a second order approximation to any distribution around the mode?
- 23
0
votes
0
answers
101
views
Approximate distribution of product of N normal i.i.d.? General case [duplicate]
Given
$N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$,
and NO assumptions about $\mu_X$ and $\sigma_X$,
looking for:
accurate closed form distribution approximation of
$Y_N=\prod\limits_{...
- 241
53
votes
4
answers
26k
views
What are the factors that cause the posterior distributions to be intractable?
In Bayesian statistics, it is often mentioned that the posterior distribution is intractable and thus approximate inference must be applied. What are the factors that cause this intractability?
- 3,637
22
votes
1
answer
5k
views
Error in normal approximation to a uniform sum distribution
One naive method for approximating a normal distribution is to add together perhaps $100$ IID random variables uniformly distributed on $[0,1]$, then recenter and rescale, relying on the Central Limit ...
- 10.8k
21
votes
1
answer
1k
views
Root finding for stochastic function
Suppose we have a function $f(x)$ that we can only observe through some noise. We can not compute $f(x)$ directly, only $f(x) + \eta$ where $\eta$ is some random noise. (In practice: I compute $f(x)$...
- 1,408
18
votes
1
answer
4k
views
Do Gaussian process (regression) have the universal approximation property?
Can any continuous function on [a, b], where a and b are real numbers, be approximated or arbitrarily close to the function (in some norm) by Gaussian Processes (Regression)?
- 755
16
votes
2
answers
11k
views
When do Taylor series approximations to expectations of (entire) functions converge?
Take an expectation of the form $E(f(X))$ for some univariate random variable $X$ and an entire function $f(\cdot)$ (i.e., the interval of convergence is the whole real line)
I have a moment ...
- 535
11
votes
2
answers
8k
views
Fast approximation to inverse Beta CDF
I am looking for a fast approximation to the inverse CDF of the Beta distribution. The approximation need not be precise, but more stress is on simplicity (I'm thinking Taylor expansion of the first 1 ...
- 12.4k
11
votes
8
answers
7k
views
Approximation of logarithm of standard normal CDF for x<0
Does anyone know of an approximation for the logarithm of the standard normal CDF for x<0?
I need to implement an algorithm that very quickly calculates it. The straightforward way, of course, is ...
- 415
10
votes
1
answer
1k
views
XGBoost: universal approximator?
There are various "universal approximation theorems" for neural networks, perhaps the most famous of which is the 1989 variant by George Cybenko. Setting aside technical conditions, the ...
- 72.9k
10
votes
2
answers
3k
views
Expectation of the softmax transform for Gaussian multivariate variables
Prelims
In the article Sequential updating of conditional probabilities on directed graphical structures by Spiegelhalter and Lauritzen they give an approximation to the expectation of a logistic ...
- 662
9
votes
1
answer
3k
views
How to understand the geometric intuition of the inner workings of neural networks?
I've been studying the theory behind ANNs lately and I wanted to understand the 'magic' behind their capability of non-linear multi-class classification. This led me to this website which does a good ...
- 15k
9
votes
1
answer
2k
views
Approximating the mathematical expectation of the argmax of a Gaussian random vector
Let $X = \left( {{X_1},...,{X_n}} \right) \sim \mathcal{N}\left( {{\mathbf{\mu }},{\mathbf{\Sigma }}} \right)$ be a Gaussian random vector and $I = \mathop {\arg \max }\limits_{i = 1,n} {X_i}$.
$I$ ...
user193726
9
votes
2
answers
3k
views
Distribution of the Levenshtein distance between two random strings
The Levenshtein or edit distance between two strings is the minimum number of edits (adding a letter, removing a letter or changing a letter) required to transform one into the other.
Assume that we ...
- 15.1k
8
votes
1
answer
1k
views
The "correct" way to approximate $\text{var}(f(X))$ via Taylor expansion
tl;dr: There are two commonly reported formulas for approximating $\text{var}(f(X))$, but one is notably better than the other. Since it isn't the "standard" Taylor expansion, where does it come from, ...
- 722
8
votes
0
answers
1k
views
Universal Approximation Theorem — Neural Networks [closed]
I have posted this question elsewhere--MSE-Meta, MSE, TCS, MetaOptimize. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer.
Universal approximation ...
- 710
8
votes
1
answer
1k
views
Analytically solving sampling with or without replacement after Poisson/Negative binomial
Short version
I am trying to analytically solve/approximate the composite likelihood that results from independent Poisson draws and further sampling with or without replacement (I don't really care ...
- 4,888
7
votes
2
answers
710
views
Bayes Factor approximation
A brute force method to approximate the Bayes Factor (the ratio of the denominators (normalizing constants) in the Bayes formula) is to do the following for the two models of interest:
repeat ...
- 741
7
votes
1
answer
2k
views
Approximating the distribution of a linear combination of beta-distributed independent random variables
This question is related with these other two questions in Cross Validated, which has been already answered:
Approximate the distribution of the sum of ind. Beta r.v
Central limit theorem when the ...
- 809
6
votes
1
answer
320
views
Continuity correction in a 2 proportion test, with different sample sizes
In a test of 2 proportions (binomial -> Normal), when the sample sizes are different, what does a continuity correction look like?
Usually, in a 1 sample test, we would divide by $n$ (sample size) ...
- 5,982
6
votes
1
answer
1k
views
Why is p(x|z) tractable but p(z|x) intractable?
In variational methods, given a set of latent variables $z$ corresponding to visible variables $x$, why is it that the probability distribution $p\left(x\middle|z\right)$ is tractable to compute, but $...
- 63
6
votes
3
answers
2k
views
Approximation of Cauchy distribution
I have a ratio of two random, (dependent or independent) normally distributed variables.
Knowing that the resulting Cauchy-distribution does not produce any moments. May I ask: Is there an ...
- 93
6
votes
1
answer
2k
views
Why do we use parametric distributions instead of empirical distributions?
The probability density function (pdf) is the first derivative of the cumulative distribution (cdf) for a continuous random variable. I take it that this only applies to well-defined distributions ...
- 4,243
6
votes
1
answer
2k
views
How should sampling ratios to estimate quantiles change with population size?
I want to cut my data of size N into k equal-sized bins. But I am happy with roughly equal-sized bins, with some $\varepsilon$ error. As precise quantiles of the data are computationally costly (...
- 997
5
votes
2
answers
9k
views
Can a Bernoulli distribution be approximated by a Normal distribution?
$$\sum_{i=1}^n bernoulli(p) = binomial(n,p) \approx \mathcal N(np, np(1-p)) = \sum_{i=1}^n \mathcal N(p, p(1-p))$$
Can I conclude that $\mathcal N(p, p(1-p))$ could represent an approximation of $...
- 51
5
votes
0
answers
3k
views
Multivariate Normal Orthant Probability
For bivariate zero-mean normal distribution $P(x_1,x_2)$, the quadrant probability is defined as $P(x_1>0,x_2>0)$ or $P(x_1<0,x_2<0).$
$P(x_1>0,x_2>0) = \frac{1}{4}+\frac{sin^{-1}(\...
- 311