2
$\begingroup$

I’m looking for standard methods (e.g. methods accepted and used by the community) for fitting a probability distribution (either a probability density function (PDF) or cumulative distribution function (CDF)) to another one. Specifically fitting to the distribution itself, not data sampled from it.

The situation is:

  • I have a distribution $g(x)$, which I can calculate by some method at any value of $x$
  • I have a second distribution $f(x;\boldsymbol{\alpha})$, where $\boldsymbol{\alpha}$ is a set of parameters defining the distribution’s shape

And I want to determine the values of $\boldsymbol{\alpha}$ that make $f(x;\boldsymbol{\alpha})$ best match $g(x)$.

I’ve looked around but not found anything particularly helpful online, which (given that this seems like a reasonable thing to want to do) might suggest I’m overthinking this. My ideas for potential approaches are:

  1. Simply fitting the functional form of $f(x;\boldsymbol{\alpha})$ to $g(x)$ using a standard method (e.g., least squares)
  2. Comparing the lowest $n$ moments of the two distributions (with $n$ set by the number of parameters in $\boldsymbol{\alpha}$)
  3. Minimising the Kullback–Leibler divergence (or some other measure of distance) between the two distributions
  4. Using the known form of $g(x)$ to pretend to generate a very large number of draws from itself (i.e., with “perfectly” distributed counts) and fitting $f(x;\boldsymbol{\alpha})$ to that synthetic dataset with a standard method.

I expect that all of these options would give very similar results if the agreement between the two distributions is very good, but perhaps not if the agreement is less good.

For context, if it’s useful, the reason I want to do something like this is that $g(x)$ is expensive to calculate, and I need a computer to calculate it many times (in different situations where it has different shapes). I’ve also noticed that it always looks very similar to a standard distribution. So, if I can do some work in advance and tabulate the parameters $\boldsymbol{\alpha}$ that can approximate it well in any situation, that would speed up my code significantly.

Any help or ideas would be much appreciated, thanks!

$\endgroup$
2
  • $\begingroup$ The most general technique that seems to fit your situation is Expectation Maximization. You mention minimizing Kullback-Leibler divergence which is fine, but you still need a search algorithm for finding parameters $\alpha$ that try to minimize KL divergence. Your problem is what all of statistical inference and machine learning focus on. The "best" technique often depends on how much data you have or how complex $g(x)$ is. I'm more familiar with fitting deep learning models to tons of data. How many parameters does $\alpha$ have or what's its dimension? $\endgroup$ Commented Dec 23, 2024 at 18:53
  • $\begingroup$ @TheOtherTerry thanks for your comment, this looks useful. Ideally I’d like a method that doesn’t care how many parameters are in $\alpha$, but in practice I wouldn’t be expecting more than $\approx 10$ parameters (certainly nowhere near 100). The example I’m playing with now only has 2, which would be easy enough to attack with brute force, but of course for more parameters something more sophisticated would be needed. $\endgroup$ Commented Jan 5 at 15:23

0

Reset to default

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.