Questions tagged [approximation]
Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.
486 questions
1
vote
0
answers
173
views
Can the error be modeled in the approximation of expectation
I have a function $s(\omega)$ that is a sum of a function with random numbers $a_m$ and looks something like the following.
$$ s(\omega) = \sum_{m = 1}^{M} f(a_m, \omega) $$ where all the $a_m$ are ...
- 325
0
votes
0
answers
117
views
approximating the density of the studentized range distribution
Is anyone aware of an approximation to the density function for the studentized range distribution https://en.wikipedia.org/wiki/Studentized_range_distribution ? I've found a fast approximation for ...
- 41
1
vote
0
answers
66
views
Restrictions on sample cumulants/moments for truncated Edgeworth expansion
I'm trying to approximate an unknown distribution by a truncated Edgeworth series, with cumulants/central moments estimated from a large sample.
I notice though that I am getting negative tail ...
- 171
1
vote
0
answers
167
views
Normal approximation to the posterior distribution
Suppose we have a posterior sample of parameter $\theta$ obtained by fitting some Bayesian model to $n$ data points.
In black is the empirical posterior density and in red is a normal approximation to ...
- 2,277
0
votes
0
answers
238
views
Overflow when computing binomial distribution for large n [duplicate]
How do you compute a binomial probability distribution for large $n$? If I try the following, I get an integer overflow in any programming language:
...
- 111
3
votes
2
answers
270
views
How should I deduce the variance and expectation of the log of a variable?
I read this paper "voom: precision weights unlock linear model analysis tools for RNA-seq read counts", in the methods, the "Delta rule for log-cpm" section:
The RNA-seq data ...
- 179
3
votes
1
answer
99
views
Robustness of Posterior distribution wrt likelihood function
Suppose we have
$$
X_1, \ldots, X_n \mid \theta \, \mathop{\sim}^{iid} \, L(\cdot \mid \theta), \quad \theta \sim \pi
$$
By Bayes' theorem, the corresponding posterior distribution is
$$
\pi_n(\mathrm ...
- 550
2
votes
2
answers
462
views
How can I use the Central Limit Theorem to calculate the distribution of $\bar{X}$?
The central limit theorem says that $$
\frac{\bar{X}-\mu}{\frac{\sigma}{\sqrt{n}}} \stackrel{\mathcal{D}}{\rightarrow} N(0,1)
$$
What is the distribution of $\bar{X}$? I've seen it given as $\sum X \...
- 151
1
vote
1
answer
135
views
Extract the functional mapping between input and output from a machine learning model
A lot of ML models, such as neural networks, are Universal Function Approximators. But when evaluating ML models, we use usually metrics, such as MSE or accuracy, to assess the performance of a ML ...
- 101
1
vote
0
answers
75
views
Do Neural Networks tend to have Zero Mean Errors in each Output?
My NN (a few linear layers with ReLUs + batch normalization, no activation in the last layer) learns to approximate vector-valued labels $y_z$ from data $z\sim\rho_z$ in a supervised way, i.e. net$(z)=...
- 111
1
vote
1
answer
245
views
Algorithm for approximating linear-interpolated curve
Goal
Given a curve defined by a set of (x, y) coordinates with linear interpolation, we want to find the best approximation using a smaller set of points (w/ linear interpolation) that fall along a ...
- 11
1
vote
1
answer
314
views
Comparing Gibbs sampler and variational inference
I am learning about variational inference and Gibbs simpler.
I am in the process of deriving variational inference on my own. In this process, I need to make a comparison with Gibbs sampler.
I am ...
- 447
0
votes
0
answers
148
views
Can $e^{E(\log(x))}$ be calculated approximating $E(\log x) $ using a second order Taylor expansion around the mean?
While searching around, I found this question
Expected value of a natural logarithm
dealing with the expected value of the natural log.
The top answer references a paper that approximates the expected ...
- 101
0
votes
0
answers
99
views
Aproximate maximum of two multivariate Gaussians with multivariate Gaussian
Given two multivariate Gaussians $G_1(\mathbf{x}), G_2(\mathbf{x})$ (not PDFs!) with the same center at the coordinate origin and different covariance matrix $\mathbf{F}_1, \mathbf{F}_2$, where $\...
- 1
4
votes
2
answers
277
views
Using 1-Layer Fully-Connected Neural Network to Appoximate Exponential Functions
Consider a 1-layer fully-connected neural network (FCNN) given by
$$
f(x) = \sum_{i=1}^n v_i\sigma\!\left({w_i}^T x\right)
$$
where $x,w_i\in\mathbb{R}^d$, $v_i\in\mathbb{R}$, and $\sigma(y)=\max(y,0)$...
- 141
0
votes
0
answers
153
views
four-point forward-difference formula using Newton's form for first order derivative [closed]
We know that ${f'(x) \approx \frac{f(x+h)- f(x)}{h}}$. If we have three points ${x_0 = x-h}$, ${x_1 = x}$, ${x_2 = x + h}$, we can compute the 3-point centered-difference formula using the Newton's ...
- 217
0
votes
0
answers
188
views
Approximate Expected Value of Product of Two Random Variables
I'm looking to find an approximation of $\mathbb{E}\left[ XY \right]$ in terms of $\mathbb{E}\left[ X \right]^n$, $\mathbb{E}\left[ Y \right]^n$, $\mathbb{E}\left[ X^n \right]$, $\mathbb{E}\left[ Y^n \...
- 111
2
votes
2
answers
300
views
Binomial to Poisson Approximation
So, a little context. The image you see is from the GCE A-LEVEL syllabus where they have defined the conditions for approximating binomial to poisson.
But why did they have mention that the ...
- 23
1
vote
0
answers
71
views
Best approximation for the size of a test
Let $X \sim \mathrm{Bernoulli}(\vartheta)$ for some unknown $\vartheta \in (0,1)$, and let $(X_1, …, X_n)$ be a moderately large IID sample for $X$.
Let $\vartheta_0 \in (0,1)$. I want to test $H_0 \...
- 111
4
votes
0
answers
970
views
Can GMM approximate any given probability density function?
I am currently studying on Bayesian models, and still new to probability theory.
I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
- 107
0
votes
1
answer
244
views
How do you define a good approximation for a probability distributions?
We know a series of probability distribution approximations that are considered good as long as some condition holds. A few examples are:
Binomial can be approximated by Normal if $np(1-p) > 10$ ...
- 455
1
vote
1
answer
100
views
How to prove the mean of a sample approximates well to the mean of the population
Suppose I have a population whose distribution is definitely not normal but both the population and sample size will be large. Is there any way I can prove/ show that the mean of a large enough sample ...
0
votes
2
answers
162
views
Approximate distribution of test statistic for weighted sample mean
Let
$$ R_{i}(t) \sim \mathcal{N}(\mu_i, \sigma_i^2), $$
denote the one period return distribution for asset $i$, from which we observe the iid samples $\{R_i(t)\}_{t=1}^{n_i}$. The MLE sample mean and ...
3
votes
2
answers
834
views
Universal approximation of Gaussians
Can gaussian kernels reproduce non continuous L2 integrable functions? ( Do non continuous L2 integrable functions lie in the RKHS constructed by a Gaussian Kernel?)
Edit:
I think my question is being ...
- 77
1
vote
0
answers
423
views
Simplifying the Kullback-Leibler divergence for a sum of distributions
I want to find an approximation of a mixture of probability distributions that minimises the Kullback-Leibler divergence (KLD).
I need to verify my result, as it seems suspect.
We have a joint ...
- 111
1
vote
0
answers
104
views
Universal Approximation Capabilities of Mixture of Weibulls
Can a mixture of $N$ Weibull distributions approximate any continuous density with non-negative support, if $N$ is sufficiently large? (If so, a reference to the proof would be greatly appreciated).
(...
- 145
2
votes
1
answer
143
views
Is there a way to correct for degrees of freedom when using a generalized linear model with a Poisson distribution featuring random effects?
I am running a generalized linear mixed effect model with a Poisson distribution to analyse count data. The model has a random effect that takes into account multiple observation obtained by the same ...
- 21
2
votes
1
answer
227
views
Confidence interval for multiple samples of ratios of counts (in R)
Data and objective
I have count data from two groups, A and B, from across multiple samples. I want to estimate the average ratio of A to B across all samples, along with a confidence interval.
Issues
...
- 75
0
votes
0
answers
54
views
What is known, in principle, about the possibility of approximating the random discrepancy between a statistical estimate and its parameter?
The difference between the value of a statistical estimate and its parameter's value is almost never exactly $0$. For example, $r - \rho$, for a unique sample $r$, is likely to be some non-zero ...
- 862
1
vote
0
answers
74
views
A nondeterministic covariance-stationary process approximated by an ARMA process
We know that the Wold Decomposition Theorem says that any purely nondeterministic covariance-stationary process, $x = [x_t : t \in \mathbb{Z}]$, can be written as a linear combination of lagged values ...
- 1,007
9
votes
1
answer
636
views
Fast likelihood evaluation for Gaussian distribution with diagonal plus low rank covariance
Let's assume the likelihood
$$
y \sim\mathcal N_p(0, \Sigma + \Lambda\Lambda^\top)
$$
where $\Sigma$ is a diagonal $p \times p$ matrix and $\Lambda$ is a $p \times d$ matrix with $d \ll p$.
What is ...
- 550
3
votes
0
answers
170
views
Number of points a one hidden layer neural-network can interpolate
We am trying to understand the number of points that a neural network of a particular size can interpolate. I think this may be isomorphic to its degree of freedom? We are not interested in whether ...
- 535
1
vote
0
answers
139
views
Connection between mean field inference and mean field theory (physics)
In variational inference, the mean-field family of probability distributions is the set of distributions that factors over its terms (i.e. each component is independent of all others). This allows us ...
- 399
4
votes
1
answer
671
views
Is there a closed form approximation for the composition of the Gamma CDF with the inverse Normal CDF?
Given $k$, $\theta$ fixed shape and scale parameters for some Gamma distribution which has a CDF $F$. Let $G^{-1}$ be the inverse CDF of the standard Normal distribution. Consider the composition $H(x)...
- 43
0
votes
0
answers
167
views
Approximation of a polynomial via histogram
Note: I originally tried to pose this question generally, without discussing the specific type of stochastic process. I hope that this can still be an interesting question generally.
Assume that we ...
3
votes
0
answers
199
views
How to solve negative binomial regression? [closed]
I want to estimate negative binomial regression for from scratch i.e. I want to write a script that will maximize maximum likelihood obtaining optimal parameters. To do so we can calculate derivatives ...
- 542
4
votes
1
answer
185
views
Approximate Posterior Predictive Quantiles with Numerical Methods
I have a posterior function which is easy to approximate using numerical methods (the posterior has only 2 parameters, and is approximately Gaussian because of the large sample). However, I need to ...
- 1,195
1
vote
0
answers
393
views
Distribution of sum of independent but not i.i.d. lognormal variables?
I am trying to find the distribution of the following variable Z:
$X_i$ are each independent with Lognormal distribution ($\mu_i, \sigma^2_i$), $X_i \in L^2$ forall $\forall i$
Z = $\sum_i cX_i$ where ...
- 143
2
votes
0
answers
188
views
approximate fisher information for intractable likelihoods
Suppose I have a data set $X_1, \ldots, X_n$, and from that I compute a statistic $T(X_1, \ldots, X_n) := T$. I want to assess how reactive/sensitive this calculation is to changes in parameter values....
- 22.4k
0
votes
0
answers
53
views
How to approximate the expression to $\sum x_i$
How to approximate the expression on the left hand side to $\sum_{i=1}^Nx_i$ as $n\to \infty$
$$ \frac{\sum\limits_{i=1}^{N}x_i^2}{n-2\frac{\sum\limits_{i=1}^{N}x_i}{N}}
\left(\sqrt{1+\frac{Nn\left(...
- 11
9
votes
3
answers
1k
views
If $X$ and $Y$ are uncorrelated random variables, then under what condition is $E[X \mid Y] \approx E[X]?$
Suppose $X$ and $Y$ are real random variables that are uncorrelated. Now, uncorrelated does not imply independence, so $E[X \mid Y] \ne E[X]$.
However, can they be said to be approximately equal? If ...
- 3,464
2
votes
1
answer
391
views
Normal approximation to Bernoulli variable
I'm looking for a normal approximation for a Bernoulli variable (so I can later sum multiple correlated approximated variables)
The trivial approximation is taking the mean and variance of the ...
- 143
0
votes
0
answers
136
views
Correlation of Financial Returns using Period-End vs. Period-Average Values
I have two time series of financial returns for assets $A$ and $B$ defined below for $n$ periods. The return $a_i$ is the percent growth in the asset price of $A$ using period-end values for $i-1$ and ...
- 31
1
vote
0
answers
152
views
How to prove that the two equations for the slope of the line of best fit are equivalent? [duplicate]
I was reading this great article on deriving the equation for the line of best fit (https://www.neelocean.com/simple-ols-estimators/), and got confused when I came across:
Rearranging:
$$\hat \beta = \...
- 111
2
votes
0
answers
157
views
How do I approximate a multivariable polynomial equation using Neural Networks?
I've been trying to experiment and test the extents to which a neural network works.
I was only able to make something with broad categorical variables function in an acceptable amount of time and in ...
- 121
1
vote
1
answer
692
views
Can I use the delta method with a function that depends on n to approximate the distribution of a function of the sum of iid random variables?
Let $X_1, X_2,...$ be i.i.d. random variables with finite mean $\mu$ and finite variance $\sigma^2$. From the Central Limit Theorem, we know that $\sqrt{n}(\bar{X_n}-\mu)$ tends in distribution to $N(...
- 107
1
vote
1
answer
411
views
Can a neural net approximate any conditional density asymptotically?
Assume that the conditional density of $ y \vert x $ is a Beta distribution for all values of x. Can a Beta distribution with parameters computed by a neural net, i.e. Beta($\hat{\alpha}$, $\hat{\beta}...
user297721
1
vote
0
answers
44
views
Why does non-parametric approach break down when the joint distribution is estimated by a finite data sample?
I am currently reading the paper on Gradient Boosting Machines - J. H. Friedman, “Greedy function approximation: A gradient boosting machine,” Ann. Stat., vol. 29, no. 5, pp. 1189–1232, 2001, doi: 10....
- 85
2
votes
1
answer
789
views
Approximating the Logit-Normal by Dirichlet
There is a known approximation of the Dirichlet Distribution by a Logit-Normal, as presented in wikipedia.
However, I am interested in the reverse, can I approximate a logit-normal by a Dirichlet?
I.e....
- 121
1
vote
0
answers
65
views
Predicting Repurchase Curves next value based on usual functional form
Some definitions first:
Acquired customers: Customers placing an order for their first time.
Cohort: Group of customers that have been acquired during the same time period.
Repurchase: An order placed ...
