Statistical Analysis of Weak Signals

PETER SONG – UNIVERSITY OF MICHIGAN

ABSTRACT

The statistical analysis of weak signals (SAWS) is a fundamental challenge in various practical domains, including questionnaire items, agrochemical residues in food, genetic variants in DNA, daily physical activity, and virus detection in wastewater. In regression analysis, identifying individual associations of weak signals is often difficult due to limited sample sizes. As a result, signals are frequently grouped into bundles to enhance detectability. Supervised homogeneity pursuit is a popular approach for forming such bundles to achieve stronger associations with outcomes of interest. Recently, we proposed a novel SAWS framework that leverages mixed-integer optimization to simultaneously perform bundle formation, association estimation, and inference.  A technical innovation pertains to the reformulation of a grouping/clustering analysis as an estimation problem. This talk will discuss both the theoretical foundations and numerical performance of this approach.

RELATED PAPERS:

• Wang, W, Wu, S, Zhu, Z, Zhou, L and Song, PXK (2024).  Supervised homogeneity fusion: A combinatorial approach. Annals of Statistics 52(1), 285-310.
• Banker, MM, Zhang, L and Song, PXK (2024). Regularized scalar-on-function regression analysis to assess functional association of critical physical activity window with biological age.  Annals of Applied Statistics 18(4): 2730-2752.
• Zhang, L, Zhang, Y, Xi, C and Song, PXK* (2024). Optimally monitoring a network of sewage manholes in infectious disease surveillance.  Statistica Sinica 35 (3).  DOI: 10.5705/ss.202022.0413