Skip to content

[config] Reject unknown optimizer names in ParamGroupConfig - #4565

Open
YeonwooSung wants to merge 1 commit into
pytorch:mainfrom
YeonwooSung:pr/validate-optimizer-name
Open

[config] Reject unknown optimizer names in ParamGroupConfig#4565
YeonwooSung wants to merge 1 commit into
pytorch:mainfrom
YeonwooSung:pr/validate-optimizer-name

Conversation

@YeonwooSung

Copy link
Copy Markdown
Contributor

Summary

ParamGroupConfig.optimizer_name accepted any string. A typo such as Adamw became NotImplementedError only after the model was built.

The allowed set is still Adam, AdamW, and DistMuon, now as a single module-level factory map. Unknown names raise ValueError when the param group is constructed.

Test plan

  • New tests construct ParamGroupConfig only (no model / container): accept the three names; reject Adamw and foo
  • CI CPU unit tests for test_optimizer_param_groups.py

A typo like Adamw only failed after model build. Validate the
closed Adam/AdamW/DistMuon set when the param group is constructed.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 10, 2026
@@ -60,12 +68,20 @@ class ParamGroupConfig:
E.g. '.*bias$', '.*norm.*', '.*\\.embed_tokens\\..*', '.*' (catch-all)"""

optimizer_name: str

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead, we should

if it sounds complicated, leave it to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants