Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models
Courtney Paquette; Elliot Paquette · 2021 · arXiv
WASTE classifies this as Negative / Null Result Report · AI classification, approximate
The study found no significant effect — useful as a negative control or null benchmark for your own design.
Abstract (excerpt)
We analyze a class of stochastic gradient algorithms with momentum on a high-dimensional random least squares problem. Our framework, inspired by random matrix theory, provides an exact (deterministic) characterization for the sequence of loss values produced by these algorithms which is expressed only in terms of the eigenvalues of the Hessian. This leads to simple expressions for nearly-optimal hyperparameters, a description of the limiting neighborhood, and average-case complexity. As a consequence, we show that (small-batch) stochastic heavy-ball momentum with a fixed momentum parameter pr
Excerpt shown for reference under fair use — read the full paper at the publisher.
About to run something similar?
Run an AI Precheck on your own design to catch failure modes like this one before you spend the time. Your first desk check is free.
Related failures
Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results*
Negative / Null Result ReportThe harmonic mean p -value for combining dependent tests
Negative / Null Result ReportGeneralizability of heterogeneous treatment effect estimates across samples
Negative / Null Result ReportNumerical predictors of arithmetic success in grades 1–6
Negative / Null Result ReportMethods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Negative / Null Result ReportShould multiple imputation be the method of choice for handling missing data in randomized trials?
WASTE indexes this work — it does not host or republish it. Failure-type classification is automated and approximate.
Metadata source: arXiv