Observed information vs. Fisher information

The observed information is also often called the observed Fisher information. It is easily confused with the Fisher information. In this article, we explore the similarities and differences between these two concepts.

Observed information

The observed information \(J(\theta)\) is the negative of the Hessian matrix (second derivative) of the log-likelihood.

Fisher information

The Fisher information \(I(\theta)\) (i.e. information) is the expected value of the observed information \(J(\theta)\). That is, \(I(\theta) = E(J(\theta))\). It is also the variance of the score, which is the gradient of the log-likelihood. It measures the information available for unknown parameters from random variable \(X\).

References:

  1. Observed information
  2. Fisher information
  3. score