According to various sources, the variance of ML estimators can be obtained from the Hessian matrix of the likelihood function. If $H$ is the Hessian of the negative log-likelihood function, then $H^{-1}$ is the variance-covariance matrix of the ML estimators.

Knowing that, we can do a simple test. We can calculate the variance of a ML estimate of a Poisson parameter using this method, and then compare the result with the correct answer that we know because, in this case, it has a closed form solution. In this way, we can see if the Hessian method is sufficiently accurate.

So, here is the code in R:

x <- rpois(500, 5)

log.likelihood <- function(lambda, x) {
  sum(x * log(lambda) - lambda - log(factorial(x)))
}

mle <- optim(4.5, function(lambda)
             -log.likelihood(if (!lambda > 0) 0.01 else lambda, x),
             method='BFGS', hessian=TRUE)

Then we can check whether $H^{-1}$ is equal to $\frac{1}{n}\text{var}(x)$ which is the actual variance of a ML Poisson parameter estimate.

> 1 / mle$hessian
           [,1]
[1,] 0.01005594
> var(x) / length(x)
[1] 0.01032308

In conclusion, it looks like while the Hessian method provides an approximation to the variance of the parameter, the approximation is somewhat off. Any suggestion about how the accuracy could be improved?

Edit

Here is a plot of the errors made by different variance estimation methods for various sample sizes. The errors appear to decrease rapidly as the sample size grows. However, when the sample is small, errors can become substantial.

The hs method uses the Hessian produced by the optimization algorithm. The richardson and complex methods use Hessian matrices estimated independently, using the hessian() function of the numDeriv package, and both produce the same result.