Skip to content

Conversation

@gronsti-amd
Copy link

@gronsti-amd gronsti-amd commented Dec 18, 2025

ROCMAiterMLASparseMetadataBuilder sets cudagraph_support, but the actual checked name is _cudagraph_support.
This causes the default value AttentionCGSupport.NEVER to be used instead of the intended AttentionCGSupport.UNIFORM_SINGLE_TOKEN_DECODE

Purpose

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

ganyi1996ppo and others added 9 commits December 11, 2025 06:09
Signed-off-by: ganyi <[email protected]>
Signed-off-by: ganyi <[email protected]>
Signed-off-by: ganyi <[email protected]>
ROCMAiterMLASparseMetadataBuilder sets cudagraph_support, but the actual
checked name is _cudagraph_support. This causes the default value
AttentionCGSupport.NEVER to be used instead of the intended
AttentionCGSupport.UNIFORM_SINGLE_TOKEN_DECODE

Signed-off-by: Stig-Arne Grönroos <[email protected]>
@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

@tvirolai-amd
Copy link

tvirolai-amd commented Dec 18, 2025

It's critical that DS3.2 has full graphs on decode. Some perf numbers: Single decode step went down from 59ms to 29ms. This is pretty much 100% speedup in this particular case on this particular node (MI300X) with a slow CPU .

There are some additional warnings in the logs so the real PR may need to retest and change things now that the graphs actually work, but they simply must work before it goes in as it's so critical for performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants