Statistical reduction before the target is known: two boundary results
Rianne de Heide
What is this note about?
What can data reduction or data collection achieve when the eventual statistical question is not known yet? This note studies two extreme boundary cases. For data already observed, exact preservation of every later finite decision problem turns out to be exactly ordinary sufficiency. For data not yet collected, a simple independent-Gaussian-stream model shows an exact additive price when any one of m coordinates may be queried only after sampling has stopped.
Abstract
Suppose that the eventual use of data is not known when the data are reduced or collected. This note considers two simple boundary cases. In a finite statistical experiment, a statistic preserves the Bayes risk for every finite later decision problem if and only if it is sufficient. Hence, when the minimal sufficient statistic is one-to-one, exact preservation of all later decision problems permits no nontrivial reduction. We then consider adaptive sampling from m independent Gaussian streams when an external query specifies the coordinate to be classified only after sampling stops. Under coordinatewise error control, the optimal symmetric average sample size is exactly m times the one-coordinate optimum. A change-of-measure argument gives the corresponding pointwise lower bound in terms of binary relative entropy.
Keywords and connections
Sufficiency; statistical decision theory; Blackwell comparison of experiments; data reduction; data compression; post-sampling target specification; adaptive sampling; sequential testing; controlled sensing; Gaussian streams; sample complexity; change of measure.