Questions tagged [approximation]
Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.
486 questions
2
votes
1
answer
334
views
CLT theorem and Berry–Esseen bounds for this special case of sampling
Consider a finite set $S=\{s_1,s_2,..s_n\}$, where $a \leq s_i\leq b$ are integers. Each element in $S$ can be chosen to a subset $S'$ in probability $p$.
We consider $n$ to be very large.
My question:...
- 153
0
votes
1
answer
291
views
What are the variations of Expectation Maximization?
To explain my question better, I will use this analogy:
In the case of the Gradient-Descent method, we have multiple variations/expansions for the main algorithm, like stochastic gradient descent (SGD)...
- 65
3
votes
1
answer
2k
views
Derivation of confidence interval of incidence rate ratio
I am trying to understand the confidence interval equation for a Incidence Rate Ratio (IRR) given several places:
$ 95\text{% CL(IRR)} = \exp(\log(\text{IRR}) \pm 1.96\times \text{SE(log(IRR))})$, ...
- 103
3
votes
1
answer
526
views
Nonlinear Iterative Partial Least Squares algorithm can calculate accurately all Principal Components?
I wanted to demonstrate a small example in order to understand better the $\textbf{Nonlinear Iterative Partial Least}$ $\textbf{Squares algorithm}$.
My goal is to calculate all the Principal ...
- 2,329
4
votes
2
answers
727
views
Universal Approximation: how does a neural network handle a ratio of inputs
Related questions/background info:
Universal Approximation Theorem — Neural Networks
Does the universal approximation theorem for neural networks hold for any activation function?
A universal ...
- 151
0
votes
1
answer
123
views
How can i find out closest lognormal distribution parameters from a GEV distributed data in R
The question is a bit weird so i'll open it up.
So i have a table of return periods for different amounts of rain. The table has been made using GEV distribution on known data and then the mean and ...
9
votes
1
answer
2k
views
Approximation of a probability distribution
I have a continuous random variable $X$ that can easily be sampled. I don't have any other assumption on $X$. Let's say I have sampled $X$ and I have constructed the set $S$. We can assume that $S$ is ...
- 193
2
votes
0
answers
146
views
Approximate a distribution as mixture of $K$ other (known, fixed) distributions
I'd like to draw samples from some "target" probability density function $f(x)$. However, I don't have a way to do that -- instead I just have access to $N$ samples, each drawn from one of $...
- 121
1
vote
0
answers
257
views
Sampling marginal distribution from joint density
Suppose we know that random vectors $x, y$ have joint density $p(x, y) \propto \exp(-U(x_1, \ldots, x_m, y_1, \ldots, y_n))$, and we want to draw a random sample from the marginal $p(x)$ (i.e. we want ...
- 111
2
votes
0
answers
250
views
Why does Kullback–Leibler divergence measure information loss when approximating a probability distribution?
I've encountered a sentence:
In information theory, Kullback–Leibler divergence is regarded as a measure of the information lost when probability distribution Q is used to approximate a true ...
- 21
2
votes
2
answers
168
views
Graphical construction of normal approximation to histogram
In Statistics by Freedman et al. it is described how to construct a normal approximation for a histogram, as follows:
Calculate mean and SD for the histogram (in the image $63.5$ and $3$ inches).
...
- 366
1
vote
0
answers
416
views
Predicting a value based solely on Correlation Coefficient
Let me set the stage.
We are dealing with two variables; $A$ and $B$.
We can easily obtain $A(x)$ for a specific data point $x$.
$B(x)$, on the other hand, is very difficult to know.
We know Pearson'...
- 11
2
votes
1
answer
431
views
Gradient based optimization of step function w.r.t number of steps
I am trying to optimize the parameter b in the following simple function using gradient descent in PyTorch:
$$ y = \frac{\lfloor{xb} \rfloor + 0.5}{b} $$
x is in $[0,1]$ and b is continuous and in $[5,...
- 123
3
votes
0
answers
76
views
Choosing training data inputs to optimize approximation
Suppose you have a smooth function $f^*:D_1 \times D_2\rightarrow\mathbb{R}$ that you observe with error as $f$ such that $$f(x,y)=f^*(x,y)+\epsilon$$ where $\epsilon$ has zero expectation (you can ...
- 1,219
1
vote
0
answers
278
views
Why KL divergence fails to approximate the means of distributions? [closed]
We have two distributions, $P$ and $Q$ such that $P$ is our input distribution and $Q$ is our target distribution. The formulation of $KL = \mathbb{E}_{P}\left[\log\frac{P}{Q}\right]$ allows us to ...
- 212
0
votes
1
answer
87
views
Optimal way to rank candidates - concrete statement
I'd like to have some statistical/probabilistic formalisations (solutions..) of the following concrete case I have heard :
"Imagine you have a set of candidates to be interviewed for a job. You ...
- 3
4
votes
1
answer
532
views
When is the ratio of two normals approximately normal?
Suppose that $X \sim N(\mu_1,\sigma_1)$ and $Y \sim N(\mu_2,\sigma_2)$ are two independent normal random variables. Define $Z = X/Y$. I noticed that there are some cases where the distribution of $Z$ ...
- 43
0
votes
0
answers
49
views
Random subsampling to approximate distribution
Suppose I have an uniform grid $A = [ a_1, a_2, a_3, \dots, a_n ]$ of $n$ points on the interval $[-c, +c]$.
As an example, consider $c=10$ and $n=10^4$ so that
$$A = [-10, -9.9979998, -9.9959996, \...
- 1,449
1
vote
0
answers
55
views
Approximation of a rational function [closed]
Suppose a function $f$ has a known form $ f(x) = \dfrac{P(x)}{Q(x)}$ where both $P,Q$ are polynomials of degree at most $d$. Assume $d$ is fairly low, take $d\leq 5$ for example.
What is the "...
- 1,449
3
votes
1
answer
515
views
Do all moments of a random variable need to be well controlled for a valid 2nd order Taylor approximation, or is the third moment sufficient?
In this post, the accepted answer states that we need certain conditions before a second order Taylor series approximation is robust, due to the fact that the variance does not control higher moments.
...
- 940
1
vote
1
answer
108
views
Given x or y can and the correlation coefficient can you approximate the other?
Give x or y can and the correlation coefficient can you approximate the other?
The definition of correlation coefficient is:
$$r=\frac{\sum(x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum(x_i-\bar{x})^2(y_i-\...
- 740
0
votes
0
answers
95
views
Can I approximate with a normal distribution?
I feel like I should know this (I graduated in physics a couple of years ago), but I'm really unsure about whether or not it's appropriate to use a normal distribution for the following case:
I have ...
0
votes
1
answer
263
views
The quality of approximation
I have $N$ random values and I initially know that it is not a Normal distribution (it is a discrete one), but it is really close to that. I estimate the expectation and variance using my number set ...
- 31
0
votes
0
answers
70
views
Low rank approximation
I'm looking for literature that deals with the following problem (does anybody know any paper related to it).
The Low-Rank Approximation problem is well known:
$$\min \|X - \hat{X}\|_{F}, \: \text{s.t....
0
votes
0
answers
60
views
Is it reasonable to look at the output of simulating from a multivariate distribution as univariate distribution? If yes, what is this called?
Suppose I have $X_{n} \sim MVN(\underline{\mu},\Sigma)$ where $n$ is large (several thousands). However, the $\mu_i's$ and the elements of $\Sigma$ are such that almost every simulation from $X_n$ ...
0
votes
1
answer
86
views
Is the set of distribution $\{ X | \max_t |f_X(t) - f_Y(t)| \leq \epsilon \}$ convex, where f is the cdf or inverse cdf?
I'm trying to figure out if the set is convex, where the maximum difference between cdf(or inverse cdf) of X and a reference distribution Y is smaller than $\epsilon$.
1.
Let $f_X(t)$ denote the cdf ...
- 33
2
votes
1
answer
829
views
How to show that normal distribution is a second order approximation to any distribution around the mode?
How can I show that normal distribution is a second order approximation to any distribution around the mode?
- 23
2
votes
1
answer
308
views
Validity of approximating a covariance matrix by making use of a probability limit?
I want to know can we approximate the covariance matrix of a random vector by making use of a probability limit.
Define the linear regression model in matrix form as
$$
\mathbf{Y} = \mathbf{X} \beta + ...
- 940
9
votes
2
answers
392
views
Trying to approximate $E[f(X)]$ - Woflram Alpha gives $E[f(X)] \approx \frac{1}{\sqrt{3}}$ but I get $E[f(X)] \approx 0$?
Let $X \sim \mathcal{N}(\mu_X,\sigma_X^2) = \mathcal{N}(0,1)$. Let $f(x) = e^{-x^2}$. I want to approximate $E[f(X)]$.
Wolfram Alpha gives
\begin{align}
E[f(X)] \approx \frac{1}{\sqrt{3}}.
\end{align}
...
- 805
0
votes
1
answer
103
views
1
vote
0
answers
48
views
Science practice: Where to introduce approximations?
In my work, I am using an algorithm which relies on estimates of the gradient of the log-posterior at a collection of Monte Carlo samples. Since this gradient is not available in closed form, I must ...
- 585
1
vote
1
answer
1k
views
Why can't we approximate the General TSP while we can approximate the Euclidean TSP? [closed]
Euclidean TSP is approximatable, whereby the triangle inequality is obeyed. However, what is the exact reason which does not allow us to approximate General TSP?
- 11
1
vote
0
answers
74
views
Approximate PDF function from "how many in each range" data
I have the following data which represent how many graduates (out of 578) have an average grade in each range:
$58$ with average grade in the range $[5, 5.99]$
$336$ with average grade in the range $[...
- 111
2
votes
1
answer
171
views
Composite priors in bayesian linear regression?
I'm not certain that "composite" is the right word for this; I've seen blogs tutorials and books that seem to link prior beliefs together. Consider MTCARS data, where miles per gallon (mpg) ...
- 3,970
1
vote
0
answers
58
views
Is it possible to go back to initial point from kth iterated point in a Newton Raphson method?
I am trying to find preimage of a kth iterated point under Newton method. Is it possible to find an initial point from which the kth iterated is derived?
- 81
6
votes
1
answer
950
views
Deriving posterior update equation in a Variational Bayes inference
I'm reading a paper (He, et al. 2010) that has used variational Bayesian inference to solve an inverse problem. I have difficulties deriving the relations for updating the variational approximations ...
- 675
6
votes
1
answer
2k
views
Why do we use parametric distributions instead of empirical distributions?
The probability density function (pdf) is the first derivative of the cumulative distribution (cdf) for a continuous random variable. I take it that this only applies to well-defined distributions ...
- 4,243
2
votes
1
answer
239
views
Weighted sum of negative binomial distributions - approximate fast parameter calculation
Let's suppose we have a convolution (weighted sum) of three negative binomials (parameterised as mean and overdispersion).
...
- 407
2
votes
0
answers
144
views
Quantifying the universal approximation theorem
Let $m\geq 1$ be an integer and $F\in \mathbb{R}[x_1, \dots, x_m]$ be a polynomial. I want to approximate $F$ on the unit hypercube $[0, 1]^m$ by a (possibly multilayer) feedforward neural network. ...
- 21
1
vote
1
answer
2k
views
Kernel approximation with Nystroem method and usage in scikit-learn
I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn.
As far as I understood, the full Gram Matrix ...
0
votes
0
answers
201
views
Maximum-likelihood histogram from noisy data
Given a sequence of noisy observations $\{x_k\in\mathbb{R}\}$ and a set of thresholds $\{t_i\in\mathbb{R}\}$ we can bin the observations using the thresholds to create a histogram.
However, since we ...
- 415
0
votes
0
answers
155
views
Approximating a distribution with an integer histogram
Given a distribution $f:[0,a)\rightarrow\mathbb{R}$, is there a simple algorithm by which to find a sequence $\{h_i\in\mathbb{N_0}\}$ such that $f(x)$ is approximated by $h_{floor(x)}$ as a histogram ...
- 415
0
votes
1
answer
96
views
How do I combine the weights of two predictor in a regression model with GRNN?
I am trying to build an algorithm that uses GRNN for regression, a model based on the formula:
My csv files are looks like:
...
1
vote
0
answers
59
views
Good list of references and books on statistical approximation, simulation and computational methods?
I am looking for books and resources that cover simulation and approximation techniques so that we do not have to follow the strict assumptions held by the many statistical models. With how fast ...
- 3,273
1
vote
1
answer
78
views
Inferring an approximate distribution for noising of data given 300,000 samples of human noising [closed]
I'm trying to find a statistical way to get an approximate distribution of all human noising. I have a dataset of over 300,000 samples of people noising words. I took basic Statistics and I would know ...
- 740
3
votes
1
answer
394
views
Proof of theorem on Poisson distribution [duplicate]
Can someone help prove this theorem? Many thanks!
If $p\to0$ and $n\to\infty$ in such a way that $\lim np = \lambda > 0$, then for $k=0, 1,\dots$:
$$\lim_{n\to\infty}\binom nkp^k (1-p)^{n-k}=\...
- 211
5
votes
3
answers
1k
views
the rules of "approximately independent"
A tutorial says
when the sample size is much smaller than the population, like 10%, we can assume that the element in sample approximately independent.
I can imagine 2 possibilities about the figure ...
- 317
2
votes
0
answers
209
views
Approximate the mean area of 2D Voronoi cell
Consider a random uniform distribution of $N$ points in $2D$ space bounded by $[0, 1]$ in both dimensions. Example:
If I want to estimate the mean area of their Voronoi cells, I have to obtain the ...
- 4,382
2
votes
1
answer
950
views
Using Gumbel distribution to approximate distribution of sample maximum --- formulae for the parameters?
Suppose you have an observable sample $X_1,...,X_n \sim \text{IID } F_X$ which has a right-tail that decreases sufficiently rapidly to apply the extreme-value theorem (e.g., a normal distribution) to ...
- 142k
8
votes
1
answer
1k
views
The "correct" way to approximate $\text{var}(f(X))$ via Taylor expansion
tl;dr: There are two commonly reported formulas for approximating $\text{var}(f(X))$, but one is notably better than the other. Since it isn't the "standard" Taylor expansion, where does it come from, ...
- 722