#Stochastic gradient descent

Articles (2)

• 🇺🇸 Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks — 19/02/2026 [USA]

  arXiv:2602.16177v1 Announce Type: cross Abstract: In this work, we propose a notion of practical learnability grounded in finite sample settings, and develop a conjugate learning theoretical framewor...

  Related: #Learning theory, #Convex optimization, #Deep neural networks, #Eigenvalue analysis
• 🇺🇸 Understanding Transformer Optimization via Gradient Heterogeneity — 19/02/2026 [USA]

  arXiv:2502.00213v4 Announce Type: replace-cross Abstract: Transformers are difficult to optimize with stochastic gradient descent (SGD) and largely rely on adaptive optimizers such as Adam. Despite t...

  Related: #Transformers, #Optimization, #Gradient heterogeneity, #Adaptive optimizers