Completeness (statistics)
In the realm of statistical inference, completeness is a desirable property of a statistic, related to its ability to uniquely identify the underlying probability distribution. A statistic is considered complete if it captures all the relevant information about the parameter of interest that is contained in the sample. More formally, completeness is defined in terms of the expected value of functions of the statistic.
Let T(X) be a statistic based on a random sample X from a distribution belonging to a parametric family {f(x; θ): θ ∈ Θ}, where Θ is the parameter space. The statistic T(X) is said to be complete if for any (measurable) function g, the following holds:
If Eθ[g(T(X))] = 0 for all θ ∈ Θ, then Pθ(g(T(X)) = 0) = 1 for all θ ∈ Θ.
In simpler terms, a complete statistic has the property that the only function of the statistic whose expected value is zero for all possible values of the parameter is the function that is identically zero (almost surely).
Key Implications and Connections:
-
Uniqueness: Completeness is closely linked to the concept of unique unbiased estimation. If a complete statistic is also unbiased for a parameter, then it is the unique minimum variance unbiased estimator (UMVUE) for that parameter. This is a powerful result, as it guarantees the best possible unbiased estimator.
-
Sufficiency: A complete sufficient statistic summarizes all the information in the sample relevant to estimating the parameter. If a statistic is complete and sufficient, it implies that no other statistic can provide any additional information about the parameter that isn't already captured by the complete sufficient statistic. This is a valuable result as it simplifies the process of statistical inference by focusing solely on the complete sufficient statistic.
-
Exponential Families: Many common statistical distributions, especially those belonging to the exponential family, have complete sufficient statistics. Identifying these statistics is crucial for efficient estimation and hypothesis testing.
-
Lehmann-Scheffé Theorem: This theorem states that if T(X) is a complete sufficient statistic for a parameter θ, and W(X) is an unbiased estimator of θ, then E[W(X)|T(X)] is the UMVUE of θ. This theorem provides a method for finding the UMVUE in many situations.
Importance of Completeness:
Completeness is a crucial concept in statistical inference because it helps to ensure the optimality of estimators. When dealing with complete statistics, we can be confident that we are using all available information from the sample to make the most accurate inferences about the underlying population parameter. While not all statistics are complete, identifying complete statistics when they exist is a significant advantage in statistical analysis.