Skip to content

use npu_moe_gating_top_k_softmax #1355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

ttanzhiqiang
Copy link
Contributor

What this PR does / why we need it?

The optimization solution for non-deepseek select_experts is to replace gating_topk_softmax with softmax+topk+to, which is optimized from 37us to 14us on bf16/fp16 of qwen3-235b
截屏2025-06-22 21 19 41
截屏2025-06-22 21 19 49

Does this PR introduce any user-facing change?

How was this patch tested?

pytest -s vllm-ascend/tests/singlecard/ops/test_gating_top_k_softmax.py

Signed-off-by: ttanzhiqiang <[email protected]>
Copy link

codecov bot commented Jun 22, 2025

Codecov Report

Attention: Patch coverage is 30.76923% with 9 lines in your changes missing coverage. Please review.

Project coverage is 27.37%. Comparing base (097e714) to head (86965fb).
Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
vllm_ascend/ops/fused_moe.py 25.00% 6 Missing ⚠️
vllm_ascend/ops/common_fused_moe.py 40.00% 3 Missing ⚠️

❌ Your patch check has failed because the patch coverage (30.76%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1355      +/-   ##
==========================================
- Coverage   27.39%   27.37%   -0.03%     
==========================================
  Files          56       56              
  Lines        6191     6196       +5     
==========================================
  Hits         1696     1696              
- Misses       4495     4500       +5     
Flag Coverage Δ
unittests 27.37% <30.76%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: ttanzhiqiang <[email protected]>
@ttanzhiqiang
Copy link
Contributor Author

@wangxiyuan @ganyi1996ppo

@ApsarasX ApsarasX added the ready read for review label Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants