Optimization on Nam Le

Recent Advances in Neural Network Optimization for LLM Training

Thu, 28 May 2026 00:00:00 +0000

The optimization landscape for LLM training looks very different from two years ago. AdamW still dominates production runs, but a wave of research is eroding that dominance from multiple angles simultaneously: matrix-aware optimizers, horizon-free schedulers, a sharply revised understanding of µP, and communication-efficient distributed methods. This post synthesizes 18 recent papers across five interconnected fronts.

The unifying thread is an active re-examination of long-held assumptions, from whether gradient geometry matters, to what µP is actually doing, to whether weight decay is a regularizer at all.

1. Muon and Non-Euclidean Optimizers #

Background #

Muon (Momentum Urthogon*alized by Newton-Schulz*) applies a gradient orthogonalization step via a Newton-Schulz iteration before each weight update. Rather than treating each parameter as an independent scalar (as Adam does), Muon recognizes that weight matrices have geometric structure and optimizes them accordingly, performing steepest descent under the spectral norm.

The core Newton-Schulz iteration, which runs stably in bfloat16 on tensor cores, is:

$$ X \leftarrow aX + b(XX^\top)X + c(XX^\top)^2 X $$

with coefficients $a = 3.4445$, $b = -4.7750$, $c = 2.0315$. In PyTorch:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


def newtonschulz5(G, steps=5, eps=1e-7):
 a, b, c = (3.4445, -4.7750, 2.0315)
 X = G.bfloat16()
 X /= (X.norm() + eps)
 if G.size(0) > G.size(1):
 X = X.T
 for _ in range(steps):
 A = X @ X.T
 B = b * A + c * A @ A
 X = a * X + B @ X
 if G.size(0) > G.size(1):
 X = X.T
 return X

A ready-to-use implementation lives at KellerJordan/Muon. Install via:

1

pip install git+https://github.com/KellerJordan/Muon

Muon is intended for hidden-layer matrix weights only. Embeddings, the output head, and scalar/vector parameters should still use AdamW:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from muon import MuonWithAuxAdam


hidden_matrix_params = [
 p for n, p in model.blocks.named_parameters()
 if p.ndim >= 2 and "embed" not in n
]
embed_params = [p for n, p in model.named_parameters() if "embed" in n]
scalar_params = [p for p in model.parameters() if p.ndim < 2]
head_params = [model.lm_head.weight]


optimizer = MuonWithAuxAdam(
 muon_params=hidden_matrix_params,
 lr=0.02,
 adamw_params=embed_params + scalar_params + head_params,
 adamw_lr=3e-4,
 adamw_wd=0.1,
)
# LR has built-in muP scaling, so no retuning is needed as you scale up

Scaling Muon: the Moonlight result #

MoonshotAI’s Moonlight (3B/16B-parameter MoE, trained on 5.7T tokens) provides the strongest evidence yet that Muon scales to real LLM training (arXiv:2502.16982, GitHub). Two fixes are needed to make Muon work beyond small scale:

Weight decay: without it, weight and output RMS norms grow until they overflow bfloat16.
Per-parameter update scale adjustment: matching the RMS update norm of AdamW by a factor of $\sqrt{(1-\beta_1)/(1+\beta_1)}$.

With these in place, scaling-law experiments indicate roughly 2× computational efficiency compared to AdamW at compute-optimal settings.

1
2
3
4
5


# Train a Qwen-like dense model with Muon (from Moonlight repo)
python3 examples/toy_train.py \
 --model qwen --optimizer muon \
 --dataset openwebtext-100k \
 --hidden_size 896 --lr 1e-3

A further efficiency variant is Flash-Muon, which reimplements the Newton-Schulz inner loop using a custom Triton kernel that exploits the symmetry of the $XX^\top$ computation, halving the effective FLOP count.

Theoretical foundations #

Kovalev (2025) shows in Understanding Gradient Orthogonalization via Non-Euclidean Trust-Region Optimization that the orthogonalized gradient update can be interpreted as a first-order trust-region method where the trust-region is defined in terms of the matrix spectral norm. This framework unifies Muon with normalized SGD and signSGD with momentum.

Pethick et al. (2025) propose Scion, a family of LMO-based algorithms that subsumes Muon, AdamW, and normalized SGD under a single framework (arXiv:2502.07529). By choosing an explicit norm for deep architectures, Scion also achieves hyperparameter transferability across model widths.

The Polar Express (Amsel et al., 2025) replaces Newton-Schulz with a minimax polar decomposition, solving a minimax problem at each iteration to minimize worst-case error. It converges faster than Newton-Schulz in both early and asymptotic stages, while remaining numerically stable in bfloat16.

Challenging the geometric narrative #

Despite the theoretical appeal, Shumaylov et al. (2026) mount a systematic challenge in Muon is Not That Special: Random or Inverted Spectra Work Just as Well. They introduce:

Freon: a family of optimizers based on Schatten (quasi-)norms, interpolating between SGD and Muon. The best-performing Schatten parameter for GPT-2 lies in the quasi-norm regime, which no LMO-based optimizer can represent.
Kaon: replaces Muon’s singular values with random noise, yet still matches Muon’s validation loss on GPT-2.

Their key insight: performance is primarily controlled by two local quantities, alignment (how well the update direction aligns with the gradient) and descent potential (step-size optimality). Muon succeeds by guaranteeing step-size optimality, not by tracking an ideal geometry.

Optimizer	Core mechanism	Key claim
Muon	Newton-Schulz orthogonalization	~2× efficiency over AdamW at compute-optimal
Scion	LMO over norm-ball	Unifies Muon/Adam; HP transferable across widths
Polar Express	Minimax polar decomposition	Faster convergence; bfloat16-safe
Freon / Kaon	Schatten quasi-norms / random SVs	Geometry is irrelevant; alignment drives performance

2. Learning Rate Scheduling #

Linear decay is provably optimal #

Defazio et al. (2023/2024) close a long-standing gap between theory and practice in Optimal Linear Decay Learning Rate Schedules and Further Refinements (arXiv:2310.07831). Under worst-case analysis, linear decay, setting $\eta_t \propto (1 - t/T)$, is the theoretically optimal schedule for a broad class of optimizers including SGD. Across 10 diverse benchmarks, it consistently outperforms cosine annealing.

$$ \eta_t = \eta_{\max} \cdot \left(1 - \frac{t}{T}\right) $$

1
2
3
4


# PyTorch built-in, the optimal default
scheduler = torch.optim.lr_scheduler.LinearLR(
 optimizer, start_factor=1.0, end_factor=0.0, total_iters=total_steps
)

The WSD cooldown phase #

The Warmup-Stable-Decay (WSD) scheduler separates training into distinct phases ending in a sharp LR drop. Dremov et al. (2025) analyse the cooldown phase specifically in Training Dynamics of the Cooldown Stage in WSD, finding:

Cooldown shapes that balance exploration and exploitation consistently outperform purely exploratory or exploitative alternatives.
There is substantial sensitivity to AdamW’s $\beta_2$ parameter during cooldown, and higher $\beta_2$ values yield consistent improvements.
Loss-landscape visualisations support the “river valley” perspective: the cooldown follows a narrow valley in parameter space.

Convex theory meets LLM practice #

Schaipp et al. (2025) show in The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training that schedules for large model training obey performance bounds from non-smooth convex optimisation. For the constant schedule with linear cooldown, the bound is:

$$ \bar{f}T - f^* \leq \frac{|x_0 - x^*|^2}{2\eta T} + \frac{\eta}{2} \sum{t=0}^{T-1} \sigma_t^2 $$

where the cooldown benefit appears explicitly through the absence of logarithmic terms. This enables principled LR transfer: exploiting the theory yields noticeable validation loss improvements for 124M and 210M Llama-type models when extending schedules for continued training.

Anytime schedules and weight averaging #

Meterez et al. (2026) prove in Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging (arXiv:2602.03702) that horizon-free (anytime) schedules exist for overparameterised linear regression, with weight averaging central to achieving minimax-optimal convergence. At 150M–300M params trained at 1–32× Chinchilla scale, a constant LR with weight averaging matches well-tuned cosine decay across the full training duration.

Weight averaging is a largely underutilised practical lever. It should be a default, not an afterthought.

ScheduleFree+ at LLM scale #

Defazio (2026) extends schedule-free learning to full LLM pretraining in ScheduleFree+: Scaling Learning-Rate-Free and Schedule-Free Learning to Large Language Models (arXiv:2605.19095). Practical fixes for large batch and model sizes enable ScheduleFree+ to achieve a 31% improvement over WSD schedules at 1000 tokens per parameter, while also providing a theoretical foundation for checkpoint merging during pretraining.

1

pip install schedulefree

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from schedulefree import AdamWScheduleFree


optimizer = AdamWScheduleFree(
 model.parameters(), lr=1e-3, warmup_steps=1000
)


# Must switch to eval mode before evaluation
optimizer.eval()
val_loss = evaluate(model)
optimizer.train()

GitHub: facebookresearch/schedule_free

3. Hyperparameter Transfer and Scaling Laws (µP) #

Weight decay as the true driver of LR transfer #

The Maximal Update Parameterisation (µP) is widely used to transfer optimal learning rates from proxy models to large ones without re-tuning. Kosson et al. (2025/2026), accepted to ICLR 2026, provide a large-scale empirical refutation of the standard µP narrative in Weight Decay May Matter More than µP for Learning Rate Transfer in Practice.

Their finding: µP’s geometric alignment assumptions, which require alignment between a layer’s inputs, weights, and gradient updates, hold only briefly at the start of training. For the remainder, it is weight decay that stabilises update dynamics across widths and facilitates LR transfer. This implies µP’s scaling primarily acts as an implicit warmup, and can be largely replaced by modified warmup schedules.

Embedding layer LR as the key factor #

Kalra & Barkeshli (2026) provide complementary evidence in Quantifying Hyperparameter Transfer and the Importance of Embedding Layer Learning Rate, tracing µP’s advantage over standard parameterisation (SP) to a single factor: the embedding layer learning rate.

In SP, the embedding LR acts as a training bottleneck. Simply increasing it by a factor of model width, matching µP, eliminates most of the gap. Three quantitative metrics are used: quality of scaling law fit, robustness to extrapolation errors, and asymptotic loss penalty.

1
2
3
4
5
6
7
8
9


# Simple fix that captures most of µP's benefit in SP
embed_lr_multiplier = model_width / base_width # = d_model / d_model_proxy


param_groups = [
 {"params": model.embed.parameters(), "lr": base_lr * embed_lr_multiplier},
 {"params": non_embed_params, "lr": base_lr},
]
optimizer = torch.optim.AdamW(param_groups, weight_decay=0.1)

Open question: Kosson et al. argue µP acts as an implicit warmup; Kalra & Barkeshli argue it is about the embedding LR. Both contradict µP’s original geometric motivation. No consensus has emerged, and the practical implications differ significantly.

4. Normalization, Weight Decay, and Variance Reduction #

The end-of-training gradient spike #

Defazio (2025) identifies a subtle pathology in Why Gradients Rapidly Increase Near the End of Training: gradient norms spike sharply near the end of long LLM runs. The diagnosis is a three-way interaction between weight decay, normalisation layers, and the LR schedule.

When a layer is followed by normalisation, its scale becomes irrelevant to the forward pass, but weight decay continues shrinking the parameters. This creates an implicit competition between the optimizer’s effective update size and normalisation rescaling, causing gradient norms to grow unchecked as the LR decays.

Fix: disable weight decay for AdamW-updated layers in architectures where those layers are directly followed by normalisation (e.g. every transformer block):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


no_wd, wd = [], []
for name, param in model.named_parameters():
 if "norm" in name or "embed" in name or param.ndim < 2:
 no_wd.append(param)
 else:
 wd.append(param)


optimizer = torch.optim.AdamW([
 {"params": wd, "weight_decay": 0.1},
 {"params": no_wd, "weight_decay": 0.0},
], lr=3e-4)

This simultaneously eliminates the spike and reduces loss throughout training. The analysis explains why weight decay should be disabled for AdamW-updated layers in architectures like modded-nanoGPT.

Weight normalisation as an alternative #

Nemotron-Flash (Fu et al., 2025, NeurIPS 2025) investigates weight normalisation as a practical mechanism in small language models, finding that it enables more effective weight updates and improves final convergence. Weight normalisation sidesteps the weight-decay/normalisation interaction described above, though at the cost of slightly worse final loss compared to a well-tuned baseline.

MARS: variance reduction meets preconditioned gradients #

Despite decades of theoretical work, variance reduction has largely failed to yield practical gains in deep learning. Yuan et al. (2024/2025) attempt to change this in MARS: Unleashing the Power of Variance Reduction for Training Large Models, proposing a unified framework that reconciles AdamW, Lion, and Shampoo with variance reduction via a scaled stochastic recursive momentum technique.

GPT-2 training results look strong. However, the comprehensive benchmark by Semenov et al. (2025), Benchmarking Optimizers for Large Language Model Pretraining, a 73-page study covering 44 figures and 48 tables across standardised scenarios, reveals that MARS does not work well with small batch sizes, limiting its practical applicability in memory-constrained settings.

This underscores the danger of evaluating optimizers on a single benchmark setup: MARS looks excellent at the batch sizes used in the original paper and brittle elsewhere.

5. Distributed Training: DiLoCo and Its Descendants #

DiLoCo (Distributed Low-Communication training) uses AdamW as an inner optimizer for $H$ local steps on each worker (typically $H = 500$), then synchronises by applying Nesterov momentum to the pseudo-gradient, the sum of all parameter changes across those inner steps. This reduces communication frequency by up to 500×.

OpenDiLoCo: the open-source foundation #

PrimeIntellect’s OpenDiLoCo provides a reproducible drop-in implementation, demonstrated training across two continents and three countries with 90–95% compute utilisation. It later served as the foundation for INTELLECT-1, a 10B-parameter model trained globally.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from functools import partial
from open_diloco.hivemind_diloco import DiLoCoOptimizer


inner_optimizer = partial(torch.optim.AdamW, lr=4e-4)
outer_optimizer = partial(
 torch.optim.SGD, lr=0.7, momentum=0.9, nesterov=True
)


optimizer = DiLoCoOptimizer(
 dht=dht,
 params=model.parameters(),
 batch_size=512,
 num_inner_steps=500, # sync every 500 steps, 500× fewer communications
 inner_optimizer=inner_optimizer,
 outer_optimizer=outer_optimizer,
)

Why DiLoCo works on a single node: SNOO #

Kallusky et al. (2025) show in SNOO: Step-K Nesterov Outer Optimizer that DiLoCo’s effectiveness, even on a single node, stems from applying Nesterov momentum to the pseudo-gradient. Their method isolates this as a standalone Lookahead variant. Results:

1.5–2.5× FLOPs efficiency gains up to $10^{23}$ training FLOPs.
Improvements increase with model size.
Compatible with both AdamW and Muon as inner optimizers.
Minimal memory overhead.

The single-worker DiLoCo achieves speedups of up to 6.32% in steps-to-loss over AdamW on a 160M Llama model.

Smoothing DiLoCo: Generalized Primal Averaging (GPA) #

Defazio et al. (2025/2026) propose GPA in Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs (arXiv:2512.17131), which decouples DiLoCo’s interpolation constants to enable smooth iterate averaging at every step, replacing uniform averaging with exponential moving averaging.

GPA unifies single-worker DiLoCo and ScheduleFree within a single non-distributed framework. Speedups over AdamW in steps-to-target-loss:

Model	Speedup
Llama-160M	8.71%
Llama-1B	10.13%
Llama-8B	9.58%

Streaming DiLoCo: towards free distributed training #

Douillard et al. (2025) address the remaining bottleneck in Streaming DiLoCo with Overlapping Communication: Towards a Distributed Free Lunch (arXiv:2501.18512): even with infrequent synchronisation, each sync exchanges all parameters simultaneously. Three fixes:

Streaming sync: synchronise only subsets of parameters at a time.
Overlapping communication: continue training during synchronisation.
Quantisation: reduce cross-worker data to fewer bits.

Together, required bandwidth drops by two orders of magnitude while maintaining comparable quality at billion-parameter scale.

Method	Setting	Key contribution	Gain
SNOO	Single-node	Nesterov momentum on pseudo-gradient	1.5–2.5× FLOP efficiency
GPA	Single-node	Smooth iterate averaging; unifies DiLoCo + SF	~9% steps-to-loss
Streaming DiLoCo	Distributed	Streaming sync + quantisation	~100× bandwidth reduction

6. Cross-Cutting Themes and Open Questions #

Several recurrent tensions emerge from reading these papers together.

Geometry vs. step-size calibration in Muon #

Kovalev, Pethick et al., and Amsel et al. offer geometric explanations for Muon’s success. Shumaylov et al. argue that geometry is practically irrelevant and step-size optimality is the true driver. Which narrative guides future research matters: geometry points toward more sophisticated matrix norms; the step-size interpretation suggests much simpler paths to similar gains.

What µP is actually doing #

Kosson et al. argue µP is primarily an implicit warmup mechanism. Kalra & Barkeshli argue it is essentially about the embedding layer LR. Both stand in contrast to µP’s original geometric motivation. The practical stakes are high: the warmup interpretation suggests µP can be discarded with a schedule change; the embedding LR interpretation suggests a single-line fix.

Weight decay as a multi-role hyperparameter #

Weight decay appears as a protagonist in three independent stories in this survey:

Defazio: source of end-of-training gradient spikes via interaction with normalisation.
Kosson et al.: the true driver of LR transfer, not µP geometry.
Kalra & Barkeshli: improves scaling law fits but hurts extrapolation robustness.

It is no longer tenable to treat weight decay as a simple regulariser with a sensible default. It must be understood per-layer and in interaction with your normalisation strategy.

DiLoCo as the practical distributed optimizer #

Despite a large body of research on distributed optimizers, DiLoCo and its derivatives appear to be the only methods that consistently add value beyond simply scaling the batch size. The finding that its benefits carry over to single-node settings (via SNOO and GPA) makes it a particularly important line of work for practitioners at all scales.

Practical Recommendations for 2026 #

Based on the convergence of evidence across these papers, for a new large training run consider:

Optimizer: Muon for hidden-layer matrix weights + AdamW for embeddings/head. The Moonlight scaling fixes (weight decay + update scale adjustment) are necessary above ~1B parameters.
Schedule: ScheduleFree+ or linear decay instead of cosine. If you need a fixed-horizon schedule, WSD with higher $\beta_2$ during cooldown.
Weight decay: Disable it for layers directly followed by normalisation to avoid end-of-training gradient spikes.
Outer optimizer: Wrap your training loop with single-worker DiLoCo (SNOO or GPA) for a ~9% efficiency gain with no architectural changes.
µP alternatives: Before adopting full µP overhead, try increasing the embedding layer LR by a factor of $d_{\text{model}} / d_{\text{proxy}}$. This may reproduce most of the benefit.

None of these require fundamental architectural changes.

References #

#	Paper	Venue	Links
1	Jordan et al. (2024): Muon: An optimizer for hidden layers	n/a	blog · GitHub
2	Liu et al. (2025): Muon is Scalable for LLM Training (Moonlight)	n/a	arXiv:2502.16982 · GitHub
3	Kovalev (2025): Understanding Gradient Orthogonalization	n/a	n/a
4	Pethick et al. (2025): Training Deep Learning Models with Norm-Constrained LMOs (Scion)	n/a	arXiv:2502.07529
5	Amsel et al. (2025): The Polar Express	n/a	n/a
6	Shumaylov et al. (2026): Muon is Not That Special (Freon/Kaon)	n/a	n/a
7	Defazio et al. (2023): Optimal Linear Decay Learning Rate Schedules	n/a	arXiv:2310.07831
8	Dremov et al. (2025): Training Dynamics of the Cooldown Stage in WSD	n/a	n/a
9	Schaipp et al. (2025): Surprising Agreement Between Convex Theory and LR Scheduling	n/a	n/a
10	Meterez et al. (2026): Anytime Pretraining	n/a	arXiv:2602.03702
11	Defazio (2026): ScheduleFree+	n/a	arXiv:2605.19095 · GitHub
12	Kosson et al. (2026): Weight Decay May Matter More than µP	ICLR 2026	n/a
13	Kalra & Barkeshli (2026): Quantifying HP Transfer and Embedding LR	n/a	n/a
14	Defazio (2025): Why Gradients Rapidly Increase Near End of Training	n/a	n/a
15	Fu et al. (2025): Nemotron-Flash	NeurIPS 2025	n/a
16	Yuan et al. (2025): MARS	n/a	n/a
17	Semenov et al. (2025): Benchmarking Optimizers for LLM Pretraining	n/a	n/a
18	Kallusky et al. (2025): SNOO	n/a	n/a
19	Defazio et al. (2026): Smoothing DiLoCo with Primal Averaging (GPA)	n/a	arXiv:2512.17131
20	Douillard et al. (2025): Streaming DiLoCo	n/a	arXiv:2501.18512
21	Douillard et al. (2023/2024): DiLoCo (original)	n/a	arXiv:2311.08105
22	PrimeIntellect AI (2024): OpenDiLoCo	n/a	GitHub · blog

Optimization Papers in JMLR Volume 26

Sun, 29 Sep 2024 00:00:00 +0000

Optimization Research Papers in JMLR Volume 25

Sun, 29 Sep 2024 00:00:00 +0000

Optimization Research Papers in JMLR Volume 25 (2024) #

This document lists papers from JMLR Volume 25 (2024) that focus on optimization research, categorized by their primary themes. Each paper is numbered starting from 1 within its subsection, with a brief description of its key contributions to optimization theory, algorithms, or applications.

Convex Optimization #

Papers addressing convex optimization problems, including sparse NMF, differential privacy, and sparse regression.

Lower Complexity Bounds of Finite-Sum Optimization Problems: The Results and Construction
Authors: Yuze Han, Guangzeng Xie, Zhihua Zhang
Description: Investigates lower complexity bounds for finite-sum optimization problems in convex settings.
Sparse NMF with Archetypal Regularization: Computational and Robustness Properties
Authors: Kayhan Behdin, Rahul Mazumder
Description: Proposes sparse non-negative matrix factorization with archetypal regularization using convex optimization.
Scaling the Convex Barrier with Sparse Dual Algorithms
Authors: Alessandro De Palma, Harkirat Singh Behl, Rudy Bunel, Philip H.S. Torr, M. Pawan Kumar
Description: Develops sparse dual algorithms for scaling convex optimization problems.
Faster Rates in Differentially Private Stochastic Convex Optimization
Authors: Jinyan Su, Lijie Hu, Di Wang
Description: Analyzes faster convergence rates for differentially private stochastic convex optimization.
Estimation of Sparse Gaussian Graphical Models with Hidden Clustering Structure
Authors: Meixia Lin, Defeng Sun, Kim-Chuan Toh, Chengjing Wang
Description: Develops convex optimization methods for sparse Gaussian graphical models with hidden clustering.
A Minimax Optimal Approach to High-Dimensional Double Sparse Linear Regression
Authors: Yanhang Zhang, Zhifan Li, Shixiang Liu, Jianxin Yin
Description: Proposes a minimax optimal approach for high-dimensional double sparse linear regression using convex optimization.
An Inexact Projected Regularized Newton Method for Fused Zero-Norms Regularization Problems
Authors: Yuqia Wu, Shaohua Pan, Xiaoqi Yang
Description: Introduces an inexact projected regularized Newton method for fused zero-norms regularization in convex optimization.

Nonconvex Optimization #

Papers tackling nonconvex optimization, focusing on ADMM, Adam-family methods, and stochastic minimax optimization.

Convergence for Nonconvex ADMM, with Applications to CT Imaging
Authors: Rina Foygel Barber, Emil Y. Sidky
Description: Studies convergence properties of nonconvex ADMM with applications to CT imaging.
Adam-Family Methods for Nonsmooth Optimization with Convergence Guarantees
Authors: Nachuan Xiao, Xiaoyin Hu, Xin Liu, Kim-Chuan Toh
Description: Develops Adam-family methods for nonsmooth nonconvex optimization with convergence guarantees.
Nonasymptotic Analysis of Stochastic Gradient Hamiltonian Monte Carlo under Local Conditions for Nonconvex Optimization
Authors: O. Deniz Akyildiz, Sotirios Sabanis
Description: Provides a nonasymptotic analysis of stochastic gradient Hamiltonian Monte Carlo for nonconvex optimization.
High Probability Convergence Bounds for Non-Convex Stochastic Gradient Descent with Sub-Weibull Noise
Authors: Liam Madden, Emiliano Dall’Anese, Stephen Becker
Description: Derives high-probability convergence bounds for nonconvex stochastic gradient descent with sub-Weibull noise.
Stochastic Regularized Majorization-Minimization with Weakly Convex and Multi-Convex Surrogates
Authors: Hanbaek Lyu
Description: Proposes stochastic regularized majorization-minimization for weakly convex and multi-convex problems.
Near-Optimal Algorithms for Stochastic Minimax Optimization
Authors: Lesi Chen, Luo Luo
Description: Develops near-optimal algorithms for stochastic minimax optimization in nonconvex settings.
Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks
Authors: Naoki Sato, Koshiro Izumi, Hideaki Iiduka
Description: Introduces a scaled conjugate gradient method for nonconvex optimization in deep neural networks.

Stochastic Optimization #

Papers focusing on stochastic optimization methods, including continuous-time approximations, momentum, and curvature estimates.

A Comparison of Continuous-Time Approximations to Stochastic Gradient Descent
Authors: Stefan Ankirchner, Stefan Perko
Description: Compares continuous-time approximations to stochastic gradient descent for optimization.
On the Generalization of Stochastic Gradient Descent with Momentum
Authors: Ali Ramezani-Kebrya, Kimon Antonakopoulos, Volkan Cevher, Ashish Khisti, Ben Liang
Description: Analyzes the generalization properties of stochastic gradient descent with momentum.
Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent
Authors: Benjamin Gess, Sebastian Kassing, Vitalii Konarovskyi
Description: Studies stochastic modified flows and mean-field limits for stochastic gradient descent dynamics.
Stochastic Approximation with Decision-Dependent Distributions: Asymptotic Normality and Optimality
Authors: Joshua Cutler, Mateo Díaz, Dmitriy Drusvyatskiy
Description: Investigates stochastic approximation with decision-dependent distributions, focusing on asymptotic normality and optimality.
An Algorithm with Optimal Dimension-Dependence for Zero-Order Nonsmooth Nonconvex Stochastic Optimization
Authors: Guy Kornowski, Ohad Shamir
Description: Proposes an algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization.
On the Hyperparameters in Stochastic Gradient Descent with Momentum
Authors: Bin Shi
Description: Examines the impact of hyperparameters in stochastic gradient descent with momentum.
Almost Sure Convergence Rates Analysis and Saddle Avoidance of Stochastic Gradient Methods
Authors: Jun Liu, Ye Yuan
Description: Analyzes almost sure convergence rates and saddle avoidance in stochastic gradient methods.
PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates
Authors: Zachary Frangella, Pratik Rathore, Shipu Zhao, Madeleine Udell
Description: Introduces preconditioned stochastic optimization methods with scalable curvature estimates.
Zeroth-Order Stochastic Approximation Algorithms for DR-Submodular Optimization
Authors: Yuefang Lian, Xiao Wang, Dachuan Xu, Zhongrui Zhao
Description: Develops zeroth-order stochastic approximation algorithms for DR-submodular optimization.
Stochastic-Constrained Stochastic Optimization with Markovian Data
Authors: Yeongjong Kim, Dabeen Lee
Description: Studies stochastic-constrained optimization with Markovian data.
High Probability and Risk-Averse Guarantees for a Stochastic Accelerated Primal-Dual Method
Authors: Yassine Laguel, Necdet Serhat Aybat, Mert Gürbüzbalaban
Description: Provides high-probability and risk-averse guarantees for a stochastic accelerated primal-dual method.

Distributed/Decentralized Optimization #

Papers addressing distributed or decentralized optimization algorithms, focusing on communication efficiency and federated learning.

Distributed Gaussian Mean Estimation under Communication Constraints: Optimal Rates and Communication-Efficient Algorithms
Authors: T. Tony Cai, Hongji Wei
Description: Develops optimal rates and communication-efficient algorithms for distributed Gaussian mean estimation.
Accelerated Gradient Tracking over Time-Varying Graphs for Decentralized Optimization
Authors: Huan Li, Zhouchen Lin
Description: Proposes accelerated gradient tracking for decentralized optimization over time-varying graphs.
Compressed and Distributed Least-Squares Regression: Convergence Rates with Applications to Federated Learning
Authors: Constantin Philippenko, Aymeric Dieuleveut
Description: Analyzes convergence rates for compressed and distributed least-squares regression in federated learning.
Federated Automatic Differentiation
Authors: Keith Rush, Zachary Charles, Zachary Garrett
Description: Introduces federated automatic differentiation for distributed optimization.
A Random Projection Approach to Personalized Federated Learning: Enhancing Communication Efficiency, Robustness, and Fairness
Authors: Yuze Han, Xiang Li, Shiyun Lin, Zhihua Zhang
Description: Proposes a random projection approach to enhance communication efficiency in personalized federated learning.
Countering the Communication Bottleneck in Federated Learning: A Highly Efficient Zero-Order Optimization Technique
Authors: Elissa Mhanna, Mohamad Assaad
Description: Develops a zero-order optimization technique to address communication bottlenecks in federated learning.

Bandits and Online Learning #

Papers addressing multi-armed bandits, online optimization, and regret minimization.

Exploration, Exploitation, and Engagement in Multi-Armed Bandits with Abandonment
Authors: Zixian Yang, Xin Liu, Lei Ying
Description: Studies exploration, exploitation, and engagement in multi-armed bandits with abandonment.
Adaptivity and Non-Stationarity: Problem-Dependent Dynamic Regret for Online Convex Optimization
Authors: Peng Zhao, Yu-Jie Zhang, Lijun Zhang, Zhi-Hua Zhou
Description: Analyzes problem-dependent dynamic regret for online convex optimization under non-stationarity.
Materials Discovery Using Max K-Armed Bandit
Authors: Nobuaki Kikkawa, Hiroshi Ohno
Description: Applies max k-armed bandit algorithms to materials discovery, focusing on regret minimization.
Finite-Time Analysis of Globally Nonstationary Multi-Armed Bandits
Authors: Junpei Komiyama, Edouard Fouché, Junya Honda
Description: Provides finite-time analysis for globally nonstationary multi-armed bandits.
Optimistic Online Mirror Descent for Bridging Stochastic and Adversarial Online Convex Optimization
Authors: Sijia Chen, Yu-Jie Zhang, Wei-Wei Tu, Peng Zhao, Lijun Zhang
Description: Develops optimistic online mirror descent for bridging stochastic and adversarial online convex optimization.
Continuous Prediction with Experts’ Advice
Authors: Nicholas J. A. Harvey, Christopher Liaw, Victor S. Portella
Description: Investigates continuous prediction with experts’ advice in online learning settings.
Regret Analysis of Bilateral Trade with a Smoothed Adversary
Authors: Nicolò Cesa-Bianchi, Tommaso Cesari, Roberto Colomboni, Federico Fusco, Stefano Leonardi
Description: Analyzes regret in bilateral trade with a smoothed adversary in online optimization.
Optimal Learning Policies for Differential Privacy in Multi-Armed Bandits
Authors: Siwei Wang, Jun Zhu
Description: Develops optimal learning policies for differential privacy in multi-armed bandits.
Information Capacity Regret Bounds for Bandits with Mediator Feedback
Authors: Khaled Eldowa, Nicolò Cesa-Bianchi, Alberto Maria Metelli, Marcello Restelli
Description: Derives regret bounds for bandits with mediator feedback, focusing on information capacity.
Contextual Bandits with Packing and Covering Constraints: A Modular Lagrangian Approach via Regression
Authors: Aleksandrs Slivkins, Xingyu Zhou, Karthik Abinav Sankararaman, Dylan J. Foster
Description: Proposes a modular Lagrangian approach for contextual bandits with packing and covering constraints.

Optimization in Reinforcement Learning #

Papers focusing on optimization techniques for reinforcement learning, including policy gradient, actor-critic, and safe RL.

Fast Policy Extragradient Methods for Competitive Games with Entropy Regularization
Authors: Shicong Cen, Yuting Wei, Yuejie Chi
Description: Develops fast policy extragradient methods for competitive games with entropy regularization in RL.
Sample-Efficient Adversarial Imitation Learning
Authors: Dahuin Jung, Hyungyu Lee, Sungroh Yoon
Description: Proposes sample-efficient adversarial imitation learning methods for RL optimization.
On the Sample Complexity and Metastability of Heavy-Tailed Policy Search in Continuous Control
Authors: Amrit Singh Bedi, Anjaly Parayil, Junyu Zhang, Mengdi Wang, Alec Koppel
Description: Analyzes sample complexity and metastability for heavy-tailed policy search in continuous control.
Off-Policy Action Anticipation in Multi-Agent Reinforcement Learning
Authors: Ariyan Bighashdel, Daan de Geus, Pavol Jancura, Gijs Dubbelman
Description: Develops off-policy action anticipation methods for multi-agent RL optimization.
Policy Gradient Methods in the Presence of Symmetries and State Abstractions
Authors: Prakash Panangaden, Sahand Rezaei-Shoshtari, Rosie Zhao, David Meger, Doina Precup
Description: Investigates policy gradient methods with symmetries and state abstractions for RL optimization.
Log Barriers for Safe Black-Box Optimization with Application to Safe Reinforcement Learning
Authors: Ilnura Usmanova, Yarden As, Maryam Kamgarpour, Andreas Krause
Description: Proposes log barriers for safe black-box optimization with applications to safe RL.
Decentralized Natural Policy Gradient with Variance Reduction for Collaborative Multi-Agent Reinforcement Learning
Authors: Jinchi Chen, Jie Feng, Weiguo Gao, Ke Wei
Description: Develops decentralized natural policy gradient with variance reduction for multi-agent RL.
Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity
Authors: Laixi Shi, Yuejie Chi
Description: Studies distributionally robust model-based offline RL with near-optimal sample complexity.
Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds
Authors: Zhenghao Xu, Xiang Ji, Minshuo Chen, Mengdi Wang, Tuo Zhao
Description: Analyzes sample complexity of neural policy mirror descent for policy optimization on low-dimensional manifolds.
Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)
Authors: Washim Uddin Mondal, Vaneet Aggarwal, Satish V. Ukkusuri
Description: Proposes mean-field approximations for cooperative constrained multi-agent RL optimization.
Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning
Authors: Luofeng Liao, Zuyue Fu, Zhuoran Yang, Yixin Wang, Dingli Ma, Mladen Kolar, Zhaoran Wang
Description: Develops instrumental variable value iteration for causal offline RL optimization.
Matryoshka Policy Gradient for Entropy-Regularized RL: Convergence and Global Optimality
Authors: François G. Ged, Maria Han Veiga
Description: Introduces a Matryoshka policy gradient method for entropy-regularized RL with convergence guarantees.
Data-Efficient Policy Evaluation Through Behavior Policy Search
Authors: Josiah P. Hanna, Yash Chandak, Philip S. Thomas, Martha White, Peter Stone, Scott Niekum
Description: Proposes data-efficient policy evaluation methods for RL through behavior policy search.
Empirical Design in Reinforcement Learning
Authors: Andrew Patterson, Samuel Neumann, Martha White, Adam White
Description: Investigates empirical design strategies for optimization in reinforcement learning.
A New, Physics-Informed Continuous-Time Reinforcement Learning Algorithm with Performance Guarantees
Authors: Brent A. Wallace, Jennie Si
Description: Develops a physics-informed continuous-time RL algorithm with performance guarantees.

Optimization Research Papers in JMLR Volume 24

Fri, 29 Sep 2023 00:00:00 +0000

Optimization Research Papers in JMLR Volume 24 (2023) #

This document lists papers from JMLR Volume 24 (2023) that focus on optimization research, categorized by their primary themes. Each paper is numbered starting from 1 within its subsection, with a brief description of its key contributions to optimization theory, algorithms, or applications.

Convex Optimization #

Papers addressing convex optimization problems, including sparse PCA, L0 regularization, and matrix decomposition.

Sparse PCA: A Geometric Approach
Authors: Dimitris Bertsimas, Driss Lahlou Kitane
Description: Develops a geometric approach for sparse principal component analysis using convex optimization techniques.
Fundamental Limits and Algorithms for Sparse Linear Regression with Sublinear Sparsity
Authors: Lan V. Truong
Description: Investigates algorithms and theoretical limits for sparse linear regression with sublinear sparsity in a convex framework.
Sparse Training with Lipschitz Continuous Loss Functions and a Weighted Group L0-norm Constraint
Authors: Michael R. Metel
Description: Proposes sparse training methods using Lipschitz continuous loss functions and group L0-norm constraints.
MARS: A Second-Order Reduction Algorithm for High-Dimensional Sparse Precision Matrices Estimation
Authors: Qian Li, Binyan Jiang, Defeng Sun
Description: Presents a second-order reduction algorithm for sparse precision matrix estimation using convex optimization.
Sparse GCA and Thresholded Gradient Descent
Authors: Sheng Gao, Zongming Ma
Description: Develops sparse generalized correlation analysis with thresholded gradient descent in a convex framework.
A Parameter-Free Conditional Gradient Method for Composite Minimization under Hölder Condition
Authors: Masaru Ito, Zhaosong Lu, Chuan He
Description: Introduces a parameter-free conditional gradient method for composite minimization under Hölder smoothness.
L0Learn: A Scalable Package for Sparse Learning using L0 Regularization
Authors: Hussein Hazimeh, Rahul Mazumder, Tim Nonet
Description: Presents a scalable package for sparse learning with L0 regularization in convex optimization.
Sparse Plus Low Rank Matrix Decomposition: A Discrete Optimization Approach
Authors: Dimitris Bertsimas, Ryan Cory-Wright, Nicholas A. G. Johnson
Description: Proposes a discrete optimization approach for sparse plus low-rank matrix decomposition using convex methods.
Distributed Sparse Regression via Penalization
Authors: Yao Ji, Gesualdo Scutari, Ying Sun, Harsha Honnappa
Description: Develops distributed sparse regression algorithms using penalization techniques in convex optimization.
Elastic Gradient Descent, an Iterative Optimization Method Approximating the Solution Paths of the Elastic Net
Authors: Oskar Allerbo, Johan Jonasson, Rebecka Jörnsten
Description: Introduces an iterative method approximating elastic net solution paths in convex settings.
A Novel Integer Linear Programming Approach for Global L0 Minimization
Authors: Diego Delle Donne, Matthieu Kowalski, Leo Liberti
Description: Proposes an integer linear programming approach for global L0 minimization in convex optimization.

Nonconvex Optimization #

Papers tackling nonconvex optimization, focusing on descent algorithms, majorization minimization, and minimax problems.

A Line-Search Descent Algorithm for Strict Saddle Functions with Complexity Guarantees
Authors: Michael J. O’Neill, Stephen J. Wright
Description: Develops a line-search descent algorithm for nonconvex strict saddle functions with complexity guarantees.
An Inertial Block Majorization Minimization Framework for Nonsmooth Nonconvex Optimization
Authors: Le Thi Khanh Hien, Duy Nhat Phan, Nicolas Gillis
Description: Proposes an inertial block majorization minimization framework for nonsmooth nonconvex optimization.
Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the O(epsilon^(-7/4)) Complexity
Authors: Huan Li, Zhouchen Lin
Description: Introduces a restarted accelerated gradient descent method for nonconvex optimization, eliminating polylogarithmic factors.
Preconditioned Gradient Descent for Overparameterized Nonconvex Burer-Monteiro Factorization with Global Optimality Certification
Authors: Gavin Zhang, Salar Fattahi, Richard Y. Zhang
Description: Develops preconditioned gradient descent for nonconvex Burer-Monteiro factorization with global optimality guarantees.
Zeroth-Order Alternating Gradient Descent Ascent Algorithms for A Class of Nonconvex-Nonconcave Minimax Problems
Authors: Zi Xu, Zi-Qi Wang, Jun-Lin Wang, Yu-Hong Dai
Description: Proposes zeroth-order alternating gradient descent ascent for nonconvex-nonconcave minimax problems.

Stochastic Optimization #

Papers focusing on stochastic optimization methods, including gradient descent, proximal point methods, and continuous-time approaches.

On the Convergence of Stochastic Gradient Descent with Bandwidth-Based Step Size
Authors: Xiaoyu Wang, Ya-xiang Yuan
Description: Analyzes convergence of stochastic gradient descent with bandwidth-based step sizes.
Stochastic Optimization under Distributional Drift
Authors: Joshua Cutler, Dmitriy Drusvyatskiy, Zaid Harchaoui
Description: Studies stochastic optimization under distributional drift with theoretical guarantees.
Improved Powered Stochastic Optimization Algorithms for Large-Scale Machine Learning
Authors: Zhuang Yang
Description: Proposes improved powered stochastic optimization algorithms for large-scale machine learning.
Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviation
Authors: Xiao-Tong Yuan, Ping Li
Description: Provides a sharper analysis of minibatch stochastic proximal point methods, focusing on stability and smoothness.
A Continuous-Time Stochastic Gradient Descent Method for Continuous Data
Authors: Kexin Jin, Jonas Latz, Chenguang Liu, Carola-Bibiane Schönlieb
Description: Introduces a continuous-time stochastic gradient descent method for continuous data optimization.
Sensitivity-Free Gradient Descent Algorithms
Authors: Ion Matei, Maksym Zhenirovskyy, Johan de Kleer, John Maxwell
Description: Develops sensitivity-free gradient descent algorithms for stochastic optimization.

Distributed/Decentralized Optimization #

Papers addressing distributed or decentralized optimization algorithms, focusing on federated learning, asynchronous updates, and network topology.

Decentralized Learning: Theoretical Optimality and Practical Improvements
Authors: Yucheng Lu, Christopher De Sa
Description: Analyzes theoretical optimality and practical improvements for decentralized learning algorithms.
A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates
Authors: Yann Fraboni, Richard Vidal, Laetitia Kameni, Marco Lorenzi
Description: Provides a general theory for federated optimization with asynchronous and heterogeneous client updates.
Buffered Asynchronous SGD for Byzantine Learning
Authors: Yi-Rui Yang, Wu-Jun Li
Description: Proposes buffered asynchronous SGD for Byzantine-resilient distributed learning.
Minimax Estimation for Personalized Federated Learning: An Alternative Between FedAvg and Local Training
Authors: Shuxiao Chen, Qinqing Zheng, Qi Long, Weijie J. Su
Description: Investigates minimax estimation for personalized federated learning, comparing FedAvg and local training.
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
Authors: Kun Yuan, Sulaiman A. Alghunaim, Xinmeng Huang
Description: Enhances decentralized SGD by addressing data heterogeneity and network topology dependence.
Multi-Consensus Decentralized Accelerated Gradient Descent
Authors: Haishan Ye, Luo Luo, Ziang Zhou, Tong Zhang
Description: Develops multi-consensus decentralized accelerated gradient descent for distributed optimization.
Accelerated Primal-Dual Mirror Dynamics for Centralized and Distributed Constrained Convex Optimization Problems
Authors: You Zhao, Xiaofeng Liao, Xing He, Mingliang Zhou, Chaojie Li
Description: Proposes accelerated primal-dual mirror dynamics for centralized and distributed convex optimization.
Beyond Spectral Gap: The Role of the Topology in Decentralized Learning
Authors: Thijs Vogels, Hadrien Hendrikx, Martin Jaggi
Description: Examines the role of network topology in decentralized learning optimization.

Bandits and Online Learning #

Papers addressing multi-armed bandits, online optimization, and regret minimization.

Adaptation to the Range in K-Armed Bandits
Authors: Hédi Hadiji, Gilles Stoltz
Description: Studies adaptation to the range in k-armed bandit problems with regret minimization.
Dimension Reduction in Contextual Online Learning via Nonparametric Variable Selection
Authors: Wenhao Li, Ningyuan Chen, L. Jeff Hong
Description: Proposes dimension reduction techniques for contextual online learning with nonparametric variable selection.
Non-Stationary Online Learning with Memory and Non-Stochastic Control
Authors: Peng Zhao, Yu-Hu Yan, Yu-Xiang Wang, Zhi-Hua Zhou
Description: Investigates non-stationary online learning with memory and non-stochastic control strategies.
Online Non-Stochastic Control with Partial Feedback
Authors: Yu-Hu Yan, Peng Zhao, Zhi-Hua Zhou
Description: Develops online non-stochastic control methods with partial feedback for optimization.
A New Look at Dynamic Regret for Non-Stationary Stochastic Bandits
Authors: Yasin Abbasi-Yadkori, András György, Nevena Lazić
Description: Analyzes dynamic regret in non-stationary stochastic bandit problems.
A PDE Approach for Regret Bounds under Partial Monitoring
Authors: Erhan Bayraktar, Ibrahim Ekren, Xin Zhang
Description: Uses a PDE-based approach to derive regret bounds for partial monitoring in online learning.
Continuous-in-Time Limit for Bayesian Bandits
Authors: Yuhua Zhu, Zachary Izzo, Lexing Ying
Description: Explores the continuous-time limit for Bayesian bandit algorithms with theoretical guarantees.
Bandit Problems with Fidelity Rewards
Authors: Gábor Lugosi, Ciara Pike-Burke, Pierre-André Savalle
Description: Studies bandit problems with fidelity rewards, focusing on regret minimization.
Linear Partial Monitoring for Sequential Decision Making: Algorithms, Regret Bounds and Applications
Authors: Johannes Kirschner, Tor Lattimore, Andreas Krause
Description: Develops algorithms and regret bounds for linear partial monitoring in sequential decision-making.

Optimization in Reinforcement Learning #

Papers focusing on optimization techniques for reinforcement learning, including actor-critic methods and constrained RL.

Reinforcement Learning for Joint Optimization of Multiple Rewards
Authors: Mridul Agarwal, Vaneet Aggarwal
Description: Focuses on reinforcement learning for optimizing multiple rewards simultaneously.
Provably Sample-Efficient Model-Free Algorithm for MDPs with Peak Constraints
Authors: Qinbo Bai, Vaneet Aggarwal, Ather Gattami
Description: Proposes a sample-efficient model-free algorithm for MDPs with peak constraints.
Off-Policy Actor-Critic with Emphatic Weightings
Authors: Eric Graves, Ehsan Imani, Raksha Kumaraswamy, Martha White
Description: Develops off-policy actor-critic methods with emphatic weightings for RL optimization.
q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity
Authors: Yanwei Jia, Xun Yu Zhou
Description: Analyzes q-learning convergence and near-optimality for MDPs with general state spaces.
Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity
Authors: Kaiqing Zhang, Sham M. Kakade, Tamer Basar, Lin F. Yang
Description: Studies model-based multi-agent RL in zero-sum Markov games with near-optimal sample complexity.
F2A2: Flexible Fully-Decentralized Approximate Actor-Critic for Cooperative Multi-Agent Reinforcement Learning
Authors: Wenhao Li, Bo Jin, Xiangfeng Wang, Junchi Yan, Hongyuan Zha
Description: Proposes a flexible fully-decentralized approximate actor-critic method for cooperative multi-agent RL.
Adaptation Augmented Model-Based Policy Optimization
Authors: Jian Shen, Hang Lai, Minghuan Liu, Han Zhao, Yong Yu, Weinan Zhang
Description: Introduces adaptation-augmented model-based policy optimization for RL.
Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees
Authors: Mo Zhou, Jianfeng Lu
Description: Develops a single timescale actor-critic method for linear quadratic regulators with convergence guarantees.
Convex Reinforcement Learning in Finite Trials
Authors: Mirco Mutti, Riccardo De Santi, Piersilvio De Bartolomeis, Marcello Restelli
Description: Investigates convex reinforcement learning with finite trials, focusing on optimization techniques.
Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
Authors: Zihao Li, Boyi Liu, Zhuoran Yang, Zhaoran Wang, Mengdi Wang
Description: Proposes a variational primal-dual policy optimization method for constrained RL.
Instance-Dependent Confidence and Early Stopping for Reinforcement Learning
Authors: Eric Xia, Koulik Khamaru, Martin J. Wainwright, Michael I. Jordan
Description: Develops instance-dependent confidence bounds and early stopping strategies for RL optimization.

Optimization Research Papers in JMLR Volume 23

Thu, 29 Sep 2022 00:00:00 +0000

Optimization Research Papers in JMLR Volume 23 (2022) #

This document lists papers from JMLR Volume 23 (2022) that focus on optimization research, categorized by their primary themes. Each paper is numbered starting from 1 within its subsection, with a brief description of its key contributions to optimization theory, algorithms, or applications.

Convex Optimization #

Papers addressing convex optimization problems, including sparse PCA, L1-regularized SVMs, and metric-constrained problems.

Solving Large-Scale Sparse PCA to Certifiable (Near) Optimality
Authors: Dimitris Bertsimas, Ryan Cory-Wright, Jean Pauphilet
Description: Develops convex optimization techniques for large-scale sparse principal component analysis with certifiable near-optimal solutions.
Novel Min-Max Reformulations of Linear Inverse Problems
Authors: Mohammed Rayyan Sheriff, Debasish Chatterjee
Description: Proposes min-max reformulations for linear inverse problems using convex optimization frameworks.
New Insights for the Multivariate Square-Root Lasso
Authors: Aaron J. Molstad
Description: Analyzes the square-root Lasso in multivariate settings, focusing on its convex optimization properties.
Towards An Efficient Approach for the Nonconvex lp Ball Projection: Algorithm and Analysis
Authors: Xiangyu Yang, Jiashan Wang, Hao Wang
Description: Develops efficient algorithms for lp ball projection, addressing both convex and nonconvex aspects.
Solving L1-Regularized SVMs and Related Linear Programs: Revisiting the Effectiveness of Column and Constraint Generation
Authors: Antoine Dedieu, Rahul Mazumder, Haoyue Wang
Description: Investigates L1-regularized SVMs using convex optimization with column and constraint generation.
Extensions to the Proximal Distance Method of Constrained Optimization
Authors: Alfonso Landeros, Oscar Hernan Madrid Padilla, Hua Zhou, Kenneth Lange
Description: Extends the proximal distance method for constrained convex optimization problems.
Stochastic Subgradient for Composite Convex Optimization with Functional Constraints
Authors: Ion Necoara, Nitesh Kumar Singh
Description: Analyzes stochastic subgradient methods for composite convex optimization with functional constraints.
On Regularized Square-Root Regression Problems: Distributionally Robust Interpretation and Fast Computations
Authors: Hong T.M. Chu, Kim-Chuan Toh, Yangjing Zhang
Description: Studies regularized square-root regression with a distributionally robust perspective and efficient computational methods.
Project and Forget: Solving Large-Scale Metric Constrained Problems
Authors: Rishi Sonthalia, Anna C. Gilbert
Description: Proposes a convex optimization approach for large-scale metric-constrained problems.
Faster Randomized Interior Point Methods for Tall/Wide Linear Programs
Authors: Agniva Chowdhury, Gregory Dexter, Palma London, Haim Avron, Petros Drineas
Description: Develops randomized interior point methods for efficient optimization of tall/wide linear programs.

Nonconvex Optimization #

Papers tackling nonconvex optimization, focusing on optimality, stability, and convergence in nonsmooth and game settings.

Optimality and Stability in Non-Convex Smooth Games
Authors: Guojun Zhang, Pascal Poupart, Yaoliang Yu
Description: Analyzes optimality and stability in nonconvex smooth games with convergence guarantees.
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization
Authors: Zhize Li, Jian Li
Description: Proposes simple and optimal stochastic gradient methods for nonsmooth, nonconvex optimization.
Oracle Complexity in Nonsmooth Nonconvex Optimization
Authors: Guy Kornowski, Ohad Shamir
Description: Studies the oracle complexity of nonsmooth nonconvex optimization problems.
Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima
Authors: Brian Swenson, Ryan Murray, H. Vincent Poor, Soummya Kar
Description: Investigates distributed SGD for nonconvex, nonsmooth optimization with convergence to local minima.

Stochastic Optimization #

Papers focusing on stochastic optimization methods, including bundle methods, zeroth-order algorithms, and adaptive techniques.

A Stochastic Bundle Method for Interpolation
Authors: Alasdair Paren, Leonard Berrada, Rudra P. K. Poudel, M. Pawan Kumar
Description: Introduces a stochastic bundle method for efficient interpolation in optimization.
On Biased Stochastic Gradient Estimation
Authors: Derek Driggs, Jingwei Liang, Carola-Bibiane Schönlieb
Description: Analyzes biases in stochastic gradient estimation and their impact on optimization performance.
Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization
Authors: Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang
Description: Proposes accelerated zeroth-order and first-order momentum methods for a range of optimization problems.
Stochastic Zeroth-Order Optimization under Nonstationarity and Nonconvexity
Authors: Abhishek Roy, Krishnakumar Balasubramanian, Saeed Ghadimi, Prasant Mohapatra
Description: Studies zeroth-order optimization in nonstationary and nonconvex settings.
Accelerating Adaptive Cubic Regularization of Newton’s Method via Random Sampling
Authors: Xi Chen, Bo Jiang, Tianyi Lin, Shuzhong Zhang
Description: Enhances Newton’s method with adaptive cubic regularization using random sampling.
A Momentumized, Adaptive, Dual Averaged Gradient Method
Authors: Aaron Defazio, Samy Jelassi
Description: Develops a momentum-based adaptive gradient method for stochastic optimization.
Stochastic DCA with Variance Reduction and Applications in Machine Learning
Authors: Hoai An Le Thi, Hoang Phuc Hau Luu, Hoai Minh Le, Tao Pham Dinh
Description: Introduces a stochastic difference-of-convex-functions algorithm with variance reduction for machine learning.
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
Authors: Alireza Fallah, Mert Gürbüzbalaban, Asuman Ozdaglar, Umut Şimşekli, Lingjiong Zhu
Description: Proposes robust stochastic gradient methods for distributed optimization in multi-agent networks.
On Acceleration for Convex Composite Minimization with Noise-Corrupted Gradients and Approximate Proximal Mapping
Authors: Qiang Zhou, Sinno Jialin Pan
Description: Addresses acceleration in convex composite minimization with noisy gradients.
Asymptotic Study of Stochastic Adaptive Algorithms in Non-Convex Landscape
Authors: Sébastien Gadat, Ioana Gavra
Description: Analyzes the asymptotic behavior of stochastic adaptive algorithms in nonconvex settings.
Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration
Authors: Congliang Chen, Li Shen, Fangyu Zou, Wei Liu
Description: Studies the Adam optimizer, focusing on nonconvexity, convergence, and mini-batch acceleration.
An Efficient Sampling Algorithm for Non-Smooth Composite Potentials
Authors: Wenlong Mou, Nicolas Flammarion, Martin J. Wainwright, Peter L. Bartlett
Description: Develops an efficient sampling algorithm for nonsmooth composite potentials in stochastic optimization.
SGD with Coordinate Sampling: Theory and Practice
Authors: Rémi Leluc, François Portier
Description: Explores coordinate sampling in stochastic gradient descent with theoretical and practical insights.

Distributed/Decentralized Optimization #

Papers addressing distributed or decentralized optimization algorithms, focusing on communication efficiency and convergence.

Asymptotic Network Independence and Step-Size for a Distributed Subgradient Method
Authors: Alex Olshevsky
Description: Analyzes step-size and convergence for a distributed subgradient optimization method.
Projection-Free Distributed Online Learning with Sublinear Communication Complexity
Authors: Yuanyu Wan, Guanghui Wang, Wei-Wei Tu, Lijun Zhang
Description: Develops projection-free algorithms for distributed online learning with reduced communication complexity.
Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization
Authors: Huan Li, Zhouchen Lin, Yongchun Fang
Description: Proposes variance-reduced methods for decentralized optimization with optimal acceleration.

Submodular Optimization #

Papers focusing on submodular optimization, particularly in model selection.

Joint Continuous and Discrete Model Selection via Submodularity
Authors: Jonathan Bunton, Paulo Tabuada
Description: Uses submodularity for joint continuous and discrete model selection in optimization.

Bandits and Online Learning #

Papers addressing multi-armed bandits, online optimization, and regret minimization.

Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism
Authors: Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, Panayotis Mertikopoulos
Description: Studies multi-agent online optimization with delays, focusing on asynchronicity and optimism.
Online Mirror Descent and Dual Averaging: Keeping Pace in the Dynamic Case
Authors: Huang Fang, Nicholas J. A. Harvey, Victor S. Portella, Michael P. Friedlander
Description: Analyzes online mirror descent and dual averaging for dynamic online optimization.
No Weighted-Regret Learning in Adversarial Bandits with Delays
Authors: Ilai Bistritz, Zhengyuan Zhou, Xi Chen, Nicholas Bambos, Jose Blanchet
Description: Investigates regret minimization in adversarial bandits with delays.
KL-UCB-Switch: Optimal Regret Bounds for Stochastic Bandits from Both a Distribution-Dependent and a Distribution-Free Viewpoints
Authors: Aurélien Garivier, Hédi Hadiji, Pierre Ménard, Gilles Stoltz
Description: Provides optimal regret bounds for stochastic bandits using KL-UCB-Switch.
Multi-Agent Multi-Armed Bandits with Limited Communication
Authors: Mridul Agarwal, Vaneet Aggarwal, Kamyar Azizzadenesheli
Description: Explores multi-agent bandits with limited communication, focusing on regret minimization.
Nonstochastic Bandits with Composite Anonymous Feedback
Authors: Nicolò Cesa-Bianchi, Tommaso Cesari, Roberto Colomboni, Claudio Gentile, Yishay Mansour
Description: Studies nonstochastic bandits with composite feedback, analyzing regret and optimization.
Expected Regret and Pseudo-Regret are Equivalent When the Optimal Arm is Unique
Authors: Daron Anderson, Douglas J. Leith
Description: Proves equivalence of expected regret and pseudo-regret in specific bandit settings.

Bayesian and Hyperparameter Optimization #

Papers addressing Bayesian optimization and hyperparameter tuning for efficient optimization.

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
Authors: Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, René Sass, Frank Hutter
Description: Presents SMAC3, a versatile Bayesian optimization package for hyperparameter tuning.
Implicit Differentiation for Fast Hyperparameter Selection in Non-Smooth Convex Learning
Authors: Quentin Bertrand, Quentin Klopfenstein, Mathurin Massias, Mathieu Blondel, Samuel Vaiter, Alexandre Gramfort, Joseph Salmon
Description: Uses implicit differentiation for efficient hyperparameter selection in nonsmooth convex optimization.
Auto-Sklearn 2.0: Hands-Free AutoML via Meta-Learning
Authors: Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, Frank Hutter
Description: Introduces Auto-Sklearn 2.0, leveraging meta-learning for automated hyperparameter optimization.

Optimization in Reinforcement Learning #

Papers focusing on optimization techniques for reinforcement learning, including policy gradient and value estimation.

A Generalized Projected Bellman Error for Off-Policy Value Estimation in Reinforcement Learning
Authors: Andrew Patterson, Adam White, Martha White
Description: Develops optimization methods for off-policy value estimation using a generalized projected Bellman error.
Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergences
Authors: Alan Chan, Hugo Silva, Sungsu Lim, Tadashi Kozuno, A. Rupam Mahmood, Martha White
Description: Investigates greedification operators for policy optimization, focusing on KL divergences.
Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms
Authors: Yanwei Jia, Xun Yu Zhou
Description: Analyzes policy gradient and actor-critic methods for continuous-time RL optimization.
On the Convergence Rates of Policy Gradient Methods
Authors: Lin Xiao
Description: Studies convergence rates of policy gradient methods in reinforcement learning.
Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor-Critic under State Distribution Mismatch
Authors: Shangtong Zhang, Remi Tachet des Combes, Romain Laroche
Description: Examines global optimality in softmax off-policy actor-critic methods under distribution mismatch.

Optimization Research Papers in JMLR Volume 22

Wed, 29 Sep 2021 00:00:00 +0000

Optimization Research Papers in JMLR Volume 22 (2021) #

This document lists papers from JMLR Volume 22 (2021) that focus on optimization research, categorized by their primary themes. Each paper is numbered starting from 1 within its subsection, with a brief description of its key contributions to optimization theory, algorithms, or applications.

Convex Optimization #

Papers addressing convex optimization problems, including clustering, Wasserstein barycenters, sparse optimization, and bandits.

Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithm
Authors: Defeng Sun, Kim-Chuan Toh, Yancheng Yuan
Description: Proposes a convex clustering model with theoretical guarantees and an efficient algorithm.
A Fast Globally Linearly Convergent Algorithm for the Computation of Wasserstein Barycenters
Authors: Lei Yang, Jia Li, Defeng Sun, Kim-Chuan Toh
Description: Develops a fast, globally linearly convergent algorithm for computing Wasserstein barycenters.
Wasserstein Barycenters Can Be Computed in Polynomial Time in Fixed Dimension
Authors: Jason M. Altschuler, Enric Boix-Adsera
Description: Demonstrates that Wasserstein barycenters can be computed in polynomial time for fixed dimensions.
From Low Probability to High Confidence in Stochastic Convex Optimization
Authors: Damek Davis, Dmitriy Drusvyatskiy, Lin Xiao, Junyu Zhang
Description: Analyzes methods to achieve high-confidence solutions in stochastic convex optimization.
Sparse and Smooth Signal Estimation: Convexification of L0-Formulations
Authors: Alper Atamturk, Andres Gomez, Shaoning Han
Description: Proposes convexification techniques for L0-formulations in sparse and smooth signal estimation.
Stochastic Proximal AUC Maximization
Authors: Yunwen Lei, Yiming Ying
Description: Develops stochastic proximal methods for maximizing the area under the ROC curve (AUC) in convex settings.
Sparse Convex Optimization via Adaptively Regularized Hard Thresholding
Authors: Kyriakos Axiotis, Maxim Sviridenko
Description: Introduces adaptively regularized hard thresholding for sparse convex optimization.
Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives
Authors: Antoine Dedieu, Hussein Hazimeh, Rahul Mazumder
Description: Explores continuous and mixed-integer optimization approaches for learning sparse classifiers.
First-Order Convergence Theory for Weakly-Convex-Weakly-Concave Min-max Problems
Authors: Mingrui Liu, Hassan Rafique, Qihang Lin, Tianbao Yang
Description: Provides first-order convergence theory for weakly convex-weakly concave min-max problems.
Convex Geometry and Duality of Over-parameterized Neural Networks
Authors: Tolga Ergen, Mert Pilanci
Description: Analyzes convex geometry and duality in over-parameterized neural networks.
Linear Bandits on Uniformly Convex Sets
Authors: Thomas Kerdreux, Christophe Roux, Alexandre d’Aspremont, Sebastian Pokutta
Description: Studies linear bandits on uniformly convex sets, focusing on convex optimization techniques.

Nonconvex Optimization #

Papers tackling nonconvex optimization, including stochastic gradient descent, neural network training, and stability properties.

Online Stochastic Gradient Descent on Non-Convex Losses from High-Dimensional Inference
Authors: Gerard Ben Arous, Reza Gheissari, Aukosh Jagannath
Description: Analyzes online stochastic gradient descent for nonconvex losses in high-dimensional inference.
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks
Authors: Henning Petzka, Cristian Sminchisescu
Description: Investigates non-attracting regions of local minima in deep and wide neural networks.
When Does Gradient Descent with Logistic Loss Find Interpolating Two-Layer Networks?
Authors: Niladri S. Chatterji, Philip M. Long, Peter L. Bartlett
Description: Examines conditions under which gradient descent with logistic loss finds interpolating two-layer networks.
Replica Exchange for Non-Convex Optimization
Authors: Jing Dong, Xin T. Tong
Description: Proposes replica exchange methods for nonconvex optimization problems.
Failures of Model-Dependent Generalization Bounds for Least-Norm Interpolation
Authors: Peter L. Bartlett, Philip M. Long
Description: Analyzes limitations of model-dependent generalization bounds in least-norm interpolation.
On the Stability Properties and the Optimization Landscape of Training Problems with Squared Loss for Neural Networks and General Nonlinear Conic Approximation Schemes
Authors: Constantin Christof
Description: Studies stability and optimization landscapes for neural network training with squared loss.

Stochastic Optimization #

Papers focusing on stochastic optimization methods, including momentum, Langevin dynamics, and communication-efficient algorithms.

Continuous Time Analysis of Momentum Methods
Authors: Nikola B. Kovachki, Andrew M. Stuart
Description: Provides a continuous-time analysis of momentum methods in stochastic optimization.
Generalization Performance of Multi-pass Stochastic Gradient Descent with Convex Loss Functions
Authors: Yunwen Lei, Ting Hu, Ke Tang
Description: Analyzes generalization performance of multi-pass stochastic gradient descent for convex losses.
High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
Authors: Wenlong Mou, Yi-An Ma, Martin J. Wainwright, Peter L. Bartlett, Michael I. Jordan
Description: Develops an accelerated MCMC algorithm using high-order Langevin diffusion.
Path Length Bounds for Gradient Descent and Flow
Authors: Chirag Gupta, Sivaraman Balakrishnan, Aaditya Ramdas
Description: Establishes path length bounds for gradient descent and flow in stochastic optimization.
Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives
Authors: Michael Muehlebach, Michael I. Jordan
Description: Analyzes momentum-based optimization from dynamical, control-theoretic, and symplectic perspectives.
L-SVRG and L-Katyusha with Arbitrary Sampling
Authors: Xun Qian, Zheng Qu, Peter Richtárik
Description: Introduces L-SVRG and L-Katyusha algorithms with arbitrary sampling for stochastic optimization.
A Lyapunov Analysis of Accelerated Methods in Optimization
Authors: Ashia C. Wilson, Ben Recht, Michael I. Jordan
Description: Provides a Lyapunov analysis for accelerated optimization methods.
NUQSGD: Provably Communication-Efficient Data-Parallel SGD via Nonuniform Quantization
Authors: Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov, Vitalii Aksenov, Dan Alistarh, Daniel M. Roy
Description: Proposes NUQSGD, a communication-efficient stochastic gradient descent method using nonuniform quantization.
An Inertial Newton Algorithm for Deep Learning
Authors: Camille Castera, Jérôme Bolte, Cédric Févotte, Edouard Pauwels
Description: Develops an inertial Newton algorithm for deep learning optimization.
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent
Authors: Tian Tong, Cong Ma, Yuejie Chi
Description: Proposes scaled gradient descent for accelerating ill-conditioned low-rank matrix estimation.
On ADMM in Deep Learning: Convergence and Saturation-Avoidance
Authors: Jinshan Zeng, Shao-Bo Lin, Yuan Yao, Ding-Xuan Zhou
Description: Analyzes convergence and saturation-avoidance properties of ADMM in deep learning.
A Unified Convergence Analysis for Shuffling-Type Gradient Methods
Authors: Lam M. Nguyen, Quoc Tran-Dinh, Dzung T. Phan, Phuong Ha Nguyen, Marten van Dijk
Description: Provides a unified convergence analysis for shuffling-type gradient methods.
Stochastic Online Optimization Using Kalman Recursion
Authors: Joseph de Vilmarest, Olivier Wintenberger
Description: Applies Kalman recursion to stochastic online optimization.
Expanding Boundaries of Gap Safe Screening
Authors: Cassio F. Dantas, Emmanuel Soubies, Cédric Févotte
Description: Expands gap safe screening techniques for stochastic optimization.
Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning
Authors: Massimo Fornasier, Lorenzo Pareschi, Hui Huang, Philippe Sünnen
Description: Develops consensus-based optimization on the sphere with applications to machine learning.
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Authors: Mert Gürbüzbalaban, Xuefeng Gao, Yuanhan Hu, Lingjiong Zhu
Description: Proposes decentralized stochastic gradient Langevin dynamics and Hamiltonian Monte Carlo methods.

Distributed/Decentralized Optimization #

Papers addressing distributed or decentralized optimization algorithms, focusing on communication efficiency and scalability.

Projection-Free Decentralized Online Learning for Submodular Maximization over Time-Varying Networks
Authors: Junlong Zhu, Qingtao Wu, Mingchuan Zhang, Ruijuan Zheng, Keqin Li
Description: Develops projection-free decentralized online learning for submodular maximization over time-varying networks.
Communication-Efficient Distributed Covariance Sketch, with Application to Distributed PCA
Authors: Zengfeng Huang, Xuemin Lin, Wenjie Zhang, Ying Zhang
Description: Proposes a communication-efficient distributed covariance sketch for distributed PCA.
Optimal Rates of Distributed Regression with Imperfect Kernels
Authors: Hongwei Sun, Qiang Wu
Description: Establishes optimal rates for distributed regression with imperfect kernels.
One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Them
Authors: Saber Salehkaleybar, Arsalan Sharifnassab, S. Jamaloddin Golestani
Description: Analyzes theoretical limits and algorithms for one-shot federated learning.
Cooperative SGD: A Unified Framework for the Design and Analysis of Local-Update SGD Algorithms
Authors: Jianyu Wang, Gauri Joshi
Description: Introduces a unified framework for designing and analyzing local-update SGD algorithms.
DeEPCA: Decentralized Exact PCA with Linear Convergence Rate
Authors: Haishan Ye, Tong Zhang
Description: Develops DeEPCA, a decentralized exact PCA method with linear convergence.

Submodular Optimization #

Papers focusing on submodular optimization, particularly in experimental design.

Batch Greedy Maximization of Non-Submodular Functions: Guarantees and Applications to Experimental Design
Authors: Jayanth Jagalur-Mohan, Youssef Marzouk
Description: Provides guarantees for batch greedy maximization of non-submodular functions with applications to experimental design.

Bandits and Online Learning #

Papers addressing multi-armed bandits, online optimization, and regret minimization.

Regulating Greed Over Time in Multi-Armed Bandits
Authors: Stefano Tracà, Cynthia Rudin, Weiyu Yan
Description: Studies methods to regulate greed over time in multi-armed bandits.
Preference-Based Online Learning with Dueling Bandits: A Survey
Authors: Viktor Bengs, Róbert Busa-Fekete, Adil El Mesaoudi-Paul, Eyke Hüllermeier
Description: Surveys preference-based online learning with dueling bandits.
On Multi-Armed Bandit Designs for Dose-Finding Trials
Authors: Maryam Aziz, Emilie Kaufmann, Marie-Karelle Riviere
Description: Explores multi-armed bandit designs for dose-finding trials.
Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Bandits
Authors: Julian Zimmert, Yevgeny Seldin
Description: Proposes Tsallis-INF, an optimal algorithm for stochastic and adversarial bandits.
Bandit Convex Optimization in Non-Stationary Environments
Authors: Peng Zhao, Guanghui Wang, Lijun Zhang, Zhi-Hua Zhou
Description: Addresses bandit convex optimization in non-stationary environments.
A Contextual Bandit Bake-off
Authors: Alberto Bietti, Alekh Agarwal, John Langford
Description: Compares contextual bandit algorithms in a comprehensive evaluation.
MetaGrad: Adaptation Using Multiple Learning Rates in Online Learning
Authors: Tim van Erven, Wouter M. Koolen, Dirk van der Hoeven
Description: Introduces MetaGrad, an adaptive online learning algorithm with multiple learning rates.
Achieving Fairness in the Stochastic Multi-Armed Bandit Problem
Authors: Vishakha Patil, Ganesh Ghalme, Vineet Nair, Y. Narahari
Description: Develops methods for achieving fairness in stochastic multi-armed bandits.
Refined Approachability Algorithms and Application to Regret Minimization with Global Costs
Authors: Joon Kwon
Description: Proposes refined approachability algorithms for regret minimization with global costs.
Bandit Learning in Decentralized Matching Markets
Authors: Lydia T. Liu, Feng Ruan, Horia Mania, Michael I. Jordan
Description: Applies bandit learning to decentralized matching markets.
Thompson Sampling Algorithms for Cascading Bandits
Authors: Zixin Zhong, Wang Chi Chueng, Vincent Y. F. Tan
Description: Develops Thompson sampling algorithms for cascading bandits.
Fast Learning for Renewal Optimization in Online Task Scheduling
Authors: Michael J. Neely
Description: Proposes fast learning methods for renewal optimization in online task scheduling.

Bayesian and Hyperparameter Optimization #

Papers addressing Bayesian optimization and hyperparameter tuning for scalable and robust optimization.

An Empirical Study of Bayesian Optimization: Acquisition Versus Partition
Authors: Erich Merrill, Alan Fern, Xiaoli Fern, Nima Dolatnia
Description: Conducts an empirical study comparing acquisition and partition strategies in Bayesian optimization.
Hyperparameter Optimization via Sequential Uniform Designs
Authors: Zebin Yang, Aijun Zhang
Description: Proposes sequential uniform designs for hyperparameter optimization.
Are We Forgetting about Compositional Optimisers in Bayesian Optimisation?
Authors: Antoine Grosnit, Alexander I. Cowen-Rivers, Rasul Tutunov, Ryan-Rhys Griffiths, Jun Wang, Haitham Bou-Ammar
Description: Explores the role of compositional optimizers in Bayesian optimization.
GIBBON: General-Purpose Information-Based Bayesian Optimisation
Authors: Henry B. Moss, David S. Leslie, Javier Gonzalez, Paul Rayson
Description: Introduces GIBBON, a general-purpose information-based Bayesian optimization framework.
On lp-Hyperparameter Learning via Bilevel Nonsmooth Optimization
Authors: Takayuki Okuno, Akiko Takeda, Akihiro Kawana, Motokazu Watanabe
Description: Studies lp-hyperparameter learning using bilevel nonsmooth optimization.

Optimization in Reinforcement Learning #

Papers focusing on optimization techniques for reinforcement learning, including policy iteration and Q-learning.

Safe Policy Iteration: A Monotonically Improving Approximate Policy Iteration Approach
Authors: Alberto Maria Metelli, Matteo Pirotta, Daniele Calandriello, Marcello Restelli
Description: Proposes a safe policy iteration method with monotonic improvement for reinforcement learning.
On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift
Authors: Alekh Agarwal, Sham M. Kakade, Jason D. Lee, Gaurav Mahajan
Description: Analyzes the optimality, approximation, and distribution shift in policy gradient methods.
Langevin Dynamics for Adaptive Inverse Reinforcement Learning of Stochastic Gradient Algorithms
Authors: Vikram Krishnamurthy, George Yin
Description: Applies Langevin dynamics to adaptive inverse reinforcement learning for stochastic gradient algorithms.
Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controls
Authors: Jeongho Kim, Jaeuk Shin, Insoon Yang
Description: Develops Hamilton-Jacobi deep Q-learning for deterministic continuous-time systems.
Partial Policy Iteration for L1-Robust Markov Decision Processes
Authors: Chin Pang Ho, Marek Petrik, Wolfram Wiesemann
Description: Introduces partial policy iteration for L1-robust Markov decision processes.
Gaussian Approximation for Bias Reduction in Q-Learning
Authors: Carlo D’Eramo, Andrea Cini, Alessandro Nuara, Matteo Pirotta, Cesare Alippi, Jan Peters, Marcello Restelli
Description: Proposes Gaussian approximation techniques for bias reduction in Q-learning.

Optimization Research Papers in JMLR Volume 21

Tue, 29 Sep 2020 00:00:00 +0000

Optimization Research Papers in JMLR Volume 21 (2020) #

This document lists papers from JMLR Volume 21 (2020) that focus on optimization research, categorized by their primary themes. Each paper is numbered starting from 1 within its subsection, with a brief description of its key contributions to optimization theory, algorithms, or applications.

Convex Optimization #

Papers addressing convex optimization problems, including complexity bounds, convergence analysis, and applications in regression and assortment optimization.

A Low Complexity Algorithm with O(√T) Regret and O(1) Constraint Violations for Online Convex Optimization with Long Term Constraints
Authors: Hao Yu, Michael J. Neely
Description: Proposes a low-complexity algorithm for online convex optimization with long-term constraints, achieving O(√T) regret and O(1) constraint violations.
Lower Bounds for Parallel and Randomized Convex Optimization
Authors: Jelena Diakonikolas, Cristóbal Guzmán
Description: Establishes lower complexity bounds for parallel and randomized algorithms in convex optimization.
Discerning the Linear Convergence of ADMM for Structured Convex Optimization through the Lens of Variational Analysis
Authors: Xiaoming Yuan, Shangzhi Zeng, Jin Zhang
Description: Analyzes the linear convergence of ADMM for structured convex optimization using variational analysis.
A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints
Authors: Qihang Lin, Selvaprabu Nadarajah, Negar Soheili, Tianbao Yang
Description: Develops a data-efficient level set method for stochastic convex optimization with expectation constraints.
Conic Optimization for Quadratic Regression Under Sparse Noise
Authors: Igor Molybog, Ramtin Madani, Javad Lavaei
Description: Applies conic optimization to quadratic regression under sparse noise conditions.
Dynamic Assortment Optimization with Changing Contextual Information
Authors: Xi Chen, Yining Wang, Yuan Zhou
Description: Addresses dynamic assortment optimization with changing contextual information using convex optimization techniques.
Convex Programming for Estimation in Nonlinear Recurrent Models
Authors: Sohail Bahmani, Justin Romberg
Description: Uses convex programming for parameter estimation in nonlinear recurrent models.

Nonconvex Optimization #

Papers tackling nonconvex optimization, focusing on guarantees for local minima, variance reduction, and algorithmic advancements.

Exact Guarantees on the Absence of Spurious Local Minima for Non-negative Rank-1 Robust Principal Component Analysis
Authors: Salar Fattahi, Somayeh Sojoudi
Description: Provides exact guarantees for the absence of spurious local minima in non-negative rank-1 robust PCA.
Stochastic Nested Variance Reduction for Nonconvex Optimization
Authors: Dongruo Zhou, Pan Xu, Quanquan Gu
Description: Introduces a stochastic nested variance reduction method for nonconvex optimization.
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization
Authors: Nhan H. Pham, Lam M. Nguyen, Dzung T. Phan, Quoc Tran-Dinh
Description: Proposes ProxSARAH, an efficient framework for stochastic composite nonconvex optimization.
Convergence Rates for the Stochastic Gradient Descent Method for Non-Convex Objective Functions
Authors: Benjamin Fehrman, Benjamin Gess, Arnulf Jentzen
Description: Analyzes convergence rates of stochastic gradient descent for nonconvex objective functions.
AdaGrad Stepsizes: Sharp Convergence Over Nonconvex Landscapes
Authors: Rachel Ward, Xiaoxia Wu, Leon Bottou
Description: Studies sharp convergence of AdaGrad stepsize schedules in nonconvex optimization.
A Sparse Semismooth Newton Based Proximal Majorization-Minimization Algorithm for Nonconvex Square-Root-Loss Regression Problems
Authors: Peipei Tang, Chengjing Wang, Defeng Sun, Kim-Chuan Toh
Description: Develops a sparse semismooth Newton-based proximal majorization-minimization algorithm for nonconvex square-root-loss regression.

Stochastic Optimization #

Papers focusing on stochastic optimization methods, including gradient descent, variance reduction, and robustness to noise.

Convergences of Regularized Algorithms and Stochastic Gradient Methods with Random Projections
Authors: Junhong Lin, Volkan Cevher
Description: Analyzes convergence of regularized algorithms and stochastic gradient methods with random projections.
Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent
Authors: Dominic Richards, Patrick Rebeschini
Description: Studies graph-dependent implicit regularization in distributed stochastic subgradient descent.
Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functions
Authors: Artin Spiridonoff, Alex Olshevsky, Ioannis Ch. Paschalidis
Description: Proposes a robust asynchronous stochastic gradient-push method with asymptotically optimal performance for strongly convex functions.
On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics
Authors: Xi Chen, Simon S. Du, Xin T. Tong
Description: Investigates stationary-point hitting time and ergodicity in stochastic gradient Langevin dynamics.
Stochastic Conditional Gradient Methods: From Convex Minimization to Submodular Maximization
Authors: Aryan Mokhtari, Hamed Hassani, Amin Karbasi
Description: Extends stochastic conditional gradient methods from convex minimization to submodular maximization.
A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning
Authors: Aryan Mokhtari, Alec Koppel, Martin Takac, Alejandro Ribeiro
Description: Introduces parallel doubly stochastic algorithms for large-scale learning.
Gradient Descent for Sparse Rank-One Matrix Completion for Crowd-Sourced Aggregation of Sparsely Interacting Workers
Authors: Yao Ma, Alex Olshevsky, Csaba Szepesvari, Venkatesh Saligrama
Description: Applies gradient descent to sparse rank-one matrix completion for crowd-sourced worker aggregation.
Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms
Authors: Junhong Lin, Volkan Cevher
Description: Establishes optimal convergence rates for distributed learning using stochastic gradient methods and spectral algorithms.
Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise
Authors: Andrei Kulunchakov, Julien Mairal
Description: Develops estimate sequences for stochastic composite optimization with variance reduction and noise robustness.
A Unified q-Memorization Framework for Asynchronous Stochastic Optimization
Authors: Bin Gu, Wenhan Xian, Zhouyuan Huo, Cheng Deng, Heng Huang
Description: Proposes a unified q-memorization framework for asynchronous stochastic optimization.
Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms
Authors: Yazhen Wang, Shang Wu
Description: Analyzes gradient descent algorithms using stochastic differential equations in statistical and computational settings.
The Error-Feedback Framework: SGD with Delayed Gradients
Authors: Sebastian U. Stich, Sai Praneeth Karimireddy
Description: Introduces an error-feedback framework for stochastic gradient descent with delayed gradients.

Distributed/Parallel Optimization #

Papers addressing distributed or parallel optimization algorithms, focusing on communication efficiency and scalability.

On the Complexity Analysis of the Primal Solutions for the Accelerated Randomized Dual Coordinate Ascent
Authors: Huan Li, Zhouchen Lin
Description: Analyzes the complexity of primal solutions for accelerated randomized dual coordinate ascent in distributed settings.
WONDER: Weighted One-shot Distributed Ridge Regression in High Dimensions
Authors: Edgar Dobriban, Yue Sheng
Description: Proposes WONDER, a weighted one-shot distributed ridge regression method for high-dimensional data.
GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning
Authors: Anis Elgabli, Jihong Park, Amrit S. Bedi, Mehdi Bennis, Vaneet Aggarwal
Description: Introduces GADMM, a fast and communication-efficient framework for distributed machine learning.
Communication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction
Authors: Boyue Li, Shicong Cen, Yuxin Chen, Yuejie Chi
Description: Develops communication-efficient distributed optimization with gradient tracking and variance reduction.
On Convergence of Distributed Approximate Newton Methods: Globalization, Sharper Bounds and Beyond
Authors: Xiao-Tong Yuan, Ping Li
Description: Analyzes convergence of distributed approximate Newton methods with sharper bounds and globalization techniques.

Submodular Optimization #

Papers focusing on submodular optimization, including minimization and maximization problems.

Quadratic Decomposable Submodular Function Minimization: Theory and Practice
Authors: Pan Li, Niao He, Olgica Milenkovic
Description: Studies quadratic decomposable submodular function minimization with theoretical and practical insights.
Optimal Algorithms for Continuous Non-monotone Submodular and DR-Submodular Maximization
Authors: Rad Niazadeh, Tim Roughgarden, Joshua R. Wang
Description: Develops optimal algorithms for continuous non-monotone submodular and DR-submodular maximization.

Bayesian and Hyperparameter Optimization #

Papers addressing Bayesian optimization and hyperparameter tuning for scalable and robust optimization.

Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
Authors: Kirthevasan Kandasamy, Karun Raju Vysyaraju, Willie Neiswanger, Biswajit Paria, Christopher R. Collins, Jeff Schneider, Barnabas Poczos, Eric P. Xing
Description: Introduces Dragonfly, a scalable and robust Bayesian optimization framework for hyperparameter tuning.
Distributionally Ambiguous Optimization for Batch Bayesian Optimization
Authors: Nikitas Rontsis, Michael A. Osborne, Paul J. Goulart
Description: Proposes distributionally ambiguous optimization for batch Bayesian optimization.
The Kalai-Smorodinsky Solution for Many-Objective Bayesian Optimization
Authors: Mickael Binois, Victor Picheny, Patrick Taillandier, Abderrahmane Habbal
Description: Applies the Kalai-Smorodinsky solution to many-objective Bayesian optimization.
Robust Reinforcement Learning with Bayesian Optimisation and Quadrature
Authors: Supratik Paul, Konstantinos Chatzilygeroudis, Kamil Ciosek, Jean-Baptiste Mouret, Michael A. Osborne, Shimon Whiteson
Description: Integrates Bayesian optimization and quadrature for robust reinforcement learning.

Optimization in Reinforcement Learning #

Papers focusing on optimization techniques for policy optimization and reinforcement learning.

Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems
Authors: Dhruv Malik, Ashwin Pananjady, Kush Bhatia, Koulik Khamaru, Peter L. Bartlett, Martin J. Wainwright
Description: Develops derivative-free methods for policy optimization in linear quadratic systems with guarantees.
Expected Policy Gradients for Reinforcement Learning
Authors: Kamil Ciosek, Shimon Whiteson
Description: Introduces expected policy gradients for reinforcement learning optimization.
Importance Sampling Techniques for Policy Optimization
Authors: Alberto Maria Metelli, Matteo Papini, Nico Montali, Marcello Restelli
Description: Proposes importance sampling techniques for efficient policy optimization in reinforcement learning.

Optimization on Nam Le

Recent Advances in Neural Network Optimization for LLM Training

1. Muon and Non-Euclidean Optimizers #

Background #

Scaling Muon: the Moonlight result #

Theoretical foundations #

Challenging the geometric narrative #

2. Learning Rate Scheduling #

Linear decay is provably optimal #

The WSD cooldown phase #

Convex theory meets LLM practice #

Anytime schedules and weight averaging #

ScheduleFree+ at LLM scale #

3. Hyperparameter Transfer and Scaling Laws (µP) #

Weight decay as the true driver of LR transfer #

Embedding layer LR as the key factor #

4. Normalization, Weight Decay, and Variance Reduction #

The end-of-training gradient spike #

Weight normalisation as an alternative #

MARS: variance reduction meets preconditioned gradients #

5. Distributed Training: DiLoCo and Its Descendants #

OpenDiLoCo: the open-source foundation #

Why DiLoCo works on a single node: SNOO #

Smoothing DiLoCo: Generalized Primal Averaging (GPA) #

Streaming DiLoCo: towards free distributed training #

6. Cross-Cutting Themes and Open Questions #

Geometry vs. step-size calibration in Muon #

What µP is actually doing #

Weight decay as a multi-role hyperparameter #

DiLoCo as the practical distributed optimizer #

Practical Recommendations for 2026 #

References #

Optimization Papers in JMLR Volume 26

Optimization Research Papers in JMLR Volume 25

Optimization Research Papers in JMLR Volume 25 (2024) #

Convex Optimization #

Nonconvex Optimization #

Stochastic Optimization #

Distributed/Decentralized Optimization #

Bandits and Online Learning #

Optimization in Reinforcement Learning #

Other Optimization Topics #

Optimization Research Papers in JMLR Volume 24

Optimization Research Papers in JMLR Volume 24 (2023) #

Convex Optimization #

Nonconvex Optimization #

Stochastic Optimization #

Distributed/Decentralized Optimization #

Bandits and Online Learning #

Optimization in Reinforcement Learning #

Other Optimization Topics #

Optimization Research Papers in JMLR Volume 23

Optimization Research Papers in JMLR Volume 23 (2022) #

Convex Optimization #

Nonconvex Optimization #

Stochastic Optimization #

Distributed/Decentralized Optimization #

Submodular Optimization #

Bandits and Online Learning #

Bayesian and Hyperparameter Optimization #

Optimization in Reinforcement Learning #

Other Optimization Topics #

Optimization Research Papers in JMLR Volume 22

Optimization Research Papers in JMLR Volume 22 (2021) #

Convex Optimization #

Nonconvex Optimization #

Stochastic Optimization #

Distributed/Decentralized Optimization #

Submodular Optimization #

Bandits and Online Learning #

Bayesian and Hyperparameter Optimization #

Optimization in Reinforcement Learning #

Other Optimization Topics #

Optimization Research Papers in JMLR Volume 21

Optimization Research Papers in JMLR Volume 21 (2020) #

Convex Optimization #

Nonconvex Optimization #

Stochastic Optimization #

Distributed/Parallel Optimization #

Submodular Optimization #