Questions tagged [approximation]
Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.
486 questions
4
votes
1
answer
92
views
Proving universal approximation through ReLU
We were discussing universal approximation theorems for neural networks and showed that the triangular function
$$
h(x) =
\begin{cases}
x+1, & x \in [-1,0] \\
1-x, & x \in [0,1] \\
0, & \...
0
votes
1
answer
97
views
Approximating the inverse of the comulative probability distribution of the normal distribution with computers
I have a sample in normal distribution, with known average ($\mu$) and deviation ($\sigma$). As it is known, its probability density function is $f=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\...
- 131
2
votes
2
answers
236
views
Approximating a Compound Poisson Distribution
I work as an engineer at a large factory, and I am trying to model the costs due to rejects each year. Given that the number of rejects ($N$) can be modelled as a Poisson distribution, and the cost of ...
2
votes
0
answers
56
views
What discussion exists regarding statisticians' relationship to statistical methodological assumptions in applications? [closed]
This is more of a philosophical question, and also a question asking for references. To those who follow statistical academic literature, there are papers that discuss philosophical issues in ...
- 1,633
1
vote
0
answers
47
views
Interpretation of similarity measures between covariance matrices, scale independence
I am working on a problem of approximating the covariance matrix of a complicated random variable. My method takes some parameters $\theta$ of the random variable and generates the covariance ...
- 2,362
1
vote
1
answer
88
views
How do we calculate truncated expectations using the Monte Carlo method?
Consider, for example, a random vector $\textbf{X}:(\Omega,\mathcal{F},\mathbb{P})\to(\mathbb{R}^{p},\mathcal{B}(\mathbb{R}^{p}))$ whose distribution is known. My question is: can we calculate the ...
- 111
3
votes
1
answer
119
views
"Clocking" biased roulette wheels - asymptotic upper bound on $(1 - \alpha)$ quantile of most frequent number probability, Ethier (2010)
Questions.
I am having difficulty working through the derivation of an asymptotic upper bound for a $(1 - \alpha)$ quantile on the probability of the most frequent number in roulette. It uses the ...
- 2,834
1
vote
0
answers
134
views
Bounding the approximation error of the expectation of the sigmoid of a gaussian
I want to bound the approximation error for $\mathbb{E}_x[\sigma(x)],$ $\sigma(x):=1/(1+\exp(-x)),$ $x\sim\mathcal{N}(\mu,v)$:
$$\mathbb{E}_x[\sigma(x)]=\sigma(\mu)+\sum_{k=1}^\infty \frac{\sigma^{(k)}...
- 243
0
votes
0
answers
139
views
Linear PDF Approximation
I am trying to estimate the shape parameter alpha of the PDF of a Pareto distribution, given that I have incomplete data. Specifically, the true dataset spans values between $10$ and $50,$ but my ...
- 117
0
votes
0
answers
62
views
How to define and train a function $f(x,y)$ with a boundary condition?
I have a dataset consisting of triplets $(x_i,y_i,z_i), i=1,2,\cdots,N$, where $x_i,y_i,z_i\in \mathbb{R}$. My goal is to find a function $f(x,y)$ that satisfies the following conditions:
Boundary ...
- 101
0
votes
0
answers
55
views
Simulating Multilevel Data to Achieve a Specific Marginal R-squared Value
I am working on simulating multilevel data with the goal of achieving a particular marginal R-squared value through simulation. Given that variance decomposition in multilevel modeling can become ...
- 399
0
votes
0
answers
86
views
Outer product approximation of derivatives of likelihood
Lately, I have been reading Muthén's paper, "Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika,...
- 21
4
votes
4
answers
1k
views
At about what number of observations does the difference in the z and t distributions become practically meaningless?
At about what number of observations does the difference in the z and t distributions become practically meaningless? Is it 30 or 100?
- 315
1
vote
0
answers
43
views
Is there a simple estimate for power transform (Box-Cox, Yeo-Yohnson) transform, that can be computed in O(n)? [closed]
So the power transform defines some function $f(\lambda,y)$ and then we're trying to find the $\lambda$ assuming that $f(\lambda|\mathbf{y}) \sim Normal(\mu,\sigma)$.
This is usually done via MLE ...
- 203
2
votes
0
answers
54
views
Any way I can practice big O notations for probability and statistics?
I have been using Taylor expansion to get some feelings and many approximating results in trying to find innovative ideas for my research. And I have seen a lot of approximate equals or asymptotically ...
- 21
1
vote
0
answers
95
views
Efficient Methods for Approximating High-Dimensional Integrals with Gaussian-Like Factors
I'm seeking a computationally efficient method to approximately evaluate high-dimensional integrals of the form:
$$\int f(\textbf{x}) \prod_i g_i(x_i) \, d\textbf{x}$$
where $f(\mathbf{x}) = (\mathbf{...
- 730
2
votes
1
answer
98
views
Estimating the success of an approximation of a known matrix
I am trying to approximate a known $N\times N$ matrix $A$ with an 'estimation' matrix $A'$. The question is, how is it possible to quantify the error in this approximation - the difference between $A$...
- 177
3
votes
1
answer
119
views
Subtraction of Monte Carlo integrals - Catastrophic cancellation
I am attempting to estimate a quantity $Q$ which is given by the difference between two functions of Monte Carlo integrals over some set of points $\{x_i\}_{i=1}^N$, call the estimator $\hat{Q}$:
$$ \...
- 414
2
votes
1
answer
207
views
Approximation of the expected value of the $i$-th standard normal order statistic in a sample of size n
For random variables $X_1, \cdots, X_n$, we denote the order statistics by
\begin{align}
X_{(1)} & = \min (X_1,\ldots, X_n) \\[6pt]
X_{(2)} & = \text{second-smallest of } X_1,\ldots, X_n \\
&...
- 155
1
vote
1
answer
141
views
Normal approximation for posterior distribution
I am reading the example 4.3.3 of "The Bayesian Choice" by Christian P. Robert and I was wondering if it is possible to obtain a normal approximation in this case to estimate the posterior. ...
- 311
6
votes
1
answer
320
views
Continuity correction in a 2 proportion test, with different sample sizes
In a test of 2 proportions (binomial -> Normal), when the sample sizes are different, what does a continuity correction look like?
Usually, in a 1 sample test, we would divide by $n$ (sample size) ...
- 5,982
0
votes
0
answers
64
views
Monte Carlo Approximation on integral of Gaussian pdf on Convex Domain
I have hard time on estimating the following integral on convex domain ($\mathcal D$) using Monte-Carlo approximation.
$$a = \int_{\mathcal D} dx f(x;\mu,\Sigma) $$
where $x \in \mathbb R^d$ and $f$ ...
- 23
0
votes
0
answers
64
views
taylor approximation multivariate OLS coefficient
Say we have the following multivariate regression model:
$ y = \beta_1 x_1 + \beta_2 x_2 + \varepsilon $
The OLS formula for the first coefficient looks like this
$ \hat{\beta}_1 = \frac{Cov(\tilde{y}...
- 220
5
votes
2
answers
212
views
Universal approximation theorem for neural networks reference
On Wikipedia, a nice theorem is given:
However, I can not find the stated theorem in the given references. So where is the stated theorem from?
- 178
1
vote
0
answers
12
views
Getting extremely poor accuracy while doing function approximation using a neural networks in PyTorch [duplicate]
I have been given a task to approximate the function 5x^3 - 10x^2 - 5x - 9 using a neural network in pytorch. The training data is the set of integers in the range [-100,100] and I have to test the ...
- 11
0
votes
0
answers
80
views
Approximation for a correlation matrix
I have a cross-correlation matrix of some parameter for each time period. E.g. expected economy growth for each months in the future, i.e. growth for Apr 2014, May 2014, ...., Dec 2018, and ...
3
votes
1
answer
115
views
What paper did Hall suggest the queuing rule of thumb $s \geq \max ( 1, \rho + \sqrt{\rho})$?
According to this site:
Hall (1991) cited an argument of his previous paper that operation research profession could and should be more scientific and less mathematical. In fact, Hall also suggested ...
- 10.1k
3
votes
1
answer
139
views
Montecarlo Confidence Interval of T distribution
Suppose:
\begin{equation}
x|\sigma^2 \sim \mathcal{N}(x; \mu, \sigma^2) \; \; st. \; \; \sigma^2 \sim \mathcal{X}^{-2}(\sigma^2; \psi, v)
\end{equation}
where $\mathcal{X}^{-2}$ is the inverse ...
- 331
1
vote
0
answers
161
views
Taylor approximation for function of a random variable [closed]
There is a function $f$ whose domain is the space of CDFs on $\mathbb{R}_+$ and whose range is $[0,1]$, e.g. $f$ maps a CDF on to a real number. Further, $f$ is continuous, increasing with respect to ...
- 111
3
votes
1
answer
141
views
Moments and PDF of solution to random quadratic equation
Consider the following random quadratic equation,
$$
x^2 + Z x + Y = 0,
$$
where,
$$
\begin{gathered}
Z \sim \mathcal{N}(\mu_Z,\sigma_Z),
\qquad
Y \sim \mathcal{N}(\mu_Y,\sigma_Y).
\end{gathered}
$$
...
- 93
1
vote
0
answers
210
views
Little's Law + Kingman's Formula --> Approximation of Expected Length of G/G/1?
Little's Law gives us
$$\mathbb{E}[L] = \lambda \mathbb{E}[W]$$
where
$L$ is the number of customers in the queue + being served
$\lambda$ is the arrival rate
$\mu$ is the service rate
$W$ is the ...
- 10.1k
3
votes
2
answers
2k
views
Diffrence in logs vs. a % changes in econometrics: why is the dif log approvimation almost always used when the exact quantity is easily available?
I have observed that in econometrics work people almost always use the difference in logs rather than the actual percentage change. This makes no sense to me. I understand that the difference in logs ...
- 3,297
3
votes
2
answers
191
views
Approximating the distribution of the product of iid beta variates
Background
I am interested in the distribution of
$$\theta_0=1-\prod_{i=1}^n(1-\theta_i)$$
where the $\theta_{i>0}$ are iid beta random variates:
$$\theta_{i>0}\sim\text{Beta}(\alpha,\beta)$$
In ...
- 1,889
5
votes
2
answers
266
views
Approximating the standard normal density with the logistic density: How to numerically optimize $\infty$-norm?
Let's say that we want to use the logistic distribution as an approximation to the standard normal density. As the location parameter of the logistic distribution is $0$, the scale parameter $s$ is ...
- 32.1k
4
votes
1
answer
161
views
Formulas or approximations for $\mathbb{E}\left( \frac{X}{\|X\|} \right)$, $X\sim N(\mu, Id)$?
This is a cross-posting of this math SE question.
I want to compute or approximate the following expected value with some analytic expression:
$\mathbb{E}\left( \frac{X}{||X||} \right)$
, where $X \in ...
- 2,362
0
votes
0
answers
43
views
Approximating a bivariate distribution with another distribution, which method to use?
Let $X \sim F(;\theta)$ and $Y \sim G(;\eta)$ be two independent continuous random variables. The greek symbols represent the parameters of those distributions. I can easily sample from these ...
- 1
2
votes
1
answer
151
views
Finding Sample Range of Fisher's z-distribution via Approximating Hypergeometric $\,_2F_1\left(\frac{1}{2},\frac{x+1}{2};\frac{3}{2};-z^2\right)$
Recently, I have encountered Hypergeometric function $\,_2 F_1\left(\frac{1}{2},\frac{x+1}{2};\frac{3}{2};-z^2\right)$ in the context of order statistics.
In particular, I am trying to evaluate an ...
- 23
0
votes
1
answer
176
views
Extrapolating a Discrete Distribution to a Continuous one
Say I have a list of the letter grades of a class (meaning some number of As, Bs, Cs, Ds and Fs). Is there any way for me to take this discrete distribution and extrapolate it to the most likely ...
- 3
2
votes
1
answer
79
views
Differential Privacy guarantee that takes into account the approximate density (e.g., the pseudo randomness) used in practice?
In theory the differential privacy guarantee comes from adding randomness to an algorithm so whatever is output is a sample from a target distribution (e.g., the Laplacian, Gaussian, Exponential ...
- 473
2
votes
1
answer
107
views
Certain approximation in the setting of three expectation values does not make sense to me
I'm currently going through some lecture notes in the field of Bayes optimization and I'm currently looking at a expression looking like this:
$$\mathbb{E}_{x^*} \left[\mathbb{E}_y\left[\left\{\mathbb{...
1
vote
0
answers
66
views
Universal approximation theorem in the $\mathcal{C}^1$-norm [closed]
Let's say we are given a one dimensional FFNN $f_N: \mathbb{R} \rightarrow \mathbb{R}$ with hidden dimension $N$
$$f_N(x)=W_1\phi(W_2x+b_2)+b_1$$
with $W_2 \in \mathbb{R}^{N },b_2 \in \mathbb{R}^{N},...
- 201
0
votes
0
answers
63
views
Approximate distribution of random variable similar to studentized mean R.V?
It is well known that the distribution of the studentized mean, i.e., $T_0 = \frac{n^{1/2} (\bar{x}- \mu)}{\left(n^{-1} \sum \limits_{i=1}^n (x_i^2 - \bar{x}^2)\right)^{1 / 2}} $, can be approximated (...
- 41
1
vote
0
answers
145
views
Could one use mixtures of Gaussians to turn MCMC posterior samples into a new prior?
Theoretically in Bayesian inference one could use one experiment's posterior as another experiment's prior, such that knowledge of the parameters accumulates from $p(\theta) \rightarrow p(\theta|\...
- 1,513
0
votes
0
answers
125
views
Which metric to compare two probability density?
I need to compare two distribution $p$ and $q$. But I don't have access to the distribution $p$, I want to approximate it by distribution $q$ that I construct iteratively by choosing design point. ...
- 1
0
votes
0
answers
154
views
Approximation on Inverse Mills ratio for the normal R.V
I've come across several approximations for Mills ratio, but I haven't found any good ones for the Inverse Mills ratio. Is there any known closed-form approximation for the Inverse Mills ratio (link) ...
- 41
2
votes
1
answer
186
views
An approximate confidence interval for the $\alpha$ parameter of a Pareto Type II distribution when $\lambda$ is known
The Pareto Type II distribution, also known as the Lomax distribution, has the following density,
$$f(x|\alpha,\lambda)=\frac{\alpha\lambda^{\alpha}}{(\lambda+x)^{\alpha+1}}, \qquad x>0,\ \alpha>...
- 57
1
vote
0
answers
65
views
Approximate X given 5 function values and y values
Given
5 Lines(table)
X values and corresponding Y1,Y2,...Y5.
How can I calculate the approximate X value given the corresponding Y's?
How can I tweak the formula if I want to weight to bias the ...
10
votes
1
answer
1k
views
XGBoost: universal approximator?
There are various "universal approximation theorems" for neural networks, perhaps the most famous of which is the 1989 variant by George Cybenko. Setting aside technical conditions, the ...
- 72.9k
3
votes
0
answers
147
views
Understanding the ridge leverage scores sampling from an arXiv paper
I give a try to read the arXiv paper Distributed Adaptive Sampling for Kernel Matrix Approximation, Calandriello et al. 2017. I got a code implementation where they compute ridge leverage scores ...
- 205
2
votes
0
answers
79
views
What are effective methods to maximize an unknown noisy function?
I have a function that takes a few hundred parameters and which returns a score I want to optimize for - It's a piece of software attempting to play a game against another player. The parameters ...
- 21