Variance estimation in compound decision theory under boundedness

Estimating variance accurately is crucial in many statistical applications, but it becomes challenging when dealing with high-dimensional data and unknown parameters, especially in the context of compound decision theory where decisions about multiple parameters are made simultaneously. Existing studies often make restrictive assumptions such as knowing the variance or imposing strong regularity conditions on the data. This paper addresses these limitations by focusing on the case where the means are bounded. Previous research lacked a sharp characterization of the minimax rate, the best possible estimation accuracy, for variance estimation in this setting.

This research paper establishes the sharp minimax rate for variance estimation under the assumption of bounded means. This means they’ve found the best possible accuracy achievable for this task, providing a key benchmark for future research. Crucially, the study demonstrates that this rate is achievable. The paper introduces a novel cumulant-based estimator, a new statistical method that leverages the unique properties of cumulants to overcome the challenges posed by unknown variances and high dimensionality. The estimator’s rate-optimality is proven mathematically. Through clever moment-matching techniques and variational representation, the authors rigorously establish the minimax lower bound, proving their estimator’s performance is optimal.