Skip to content

[cutedsl] Search the KV tile width and seed the sm_103 dense shapes at 160 - #3576

Open
yushangdi wants to merge 1 commit into
yushangdi/stack/110from
yushangdi/stack/111
Open

[cutedsl] Search the KV tile width and seed the sm_103 dense shapes at 160#3576
yushangdi wants to merge 1 commit into
yushangdi/stack/110from
yushangdi/stack/111

Conversation

@yushangdi

@yushangdi yushangdi commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Stacked PRs:


[cutedsl] Search the KV tile width and seed the sm_103 dense shapes at 160

Add 160 to the searchable KV tile widths and promote it in the sm_103 dense
policy, together with the deeper staging (kv_stage 6) the wider tile leaves
room for. The choice set is deliberately constant across sequence lengths:
gating it on divisibility would make the search surface length-dependent,
and the masked trailing tile makes the width legal at any length anyway.

Dense 256K also moves off packed exp2 onto the resident softmax, which is
what the search now finds for it once the seed cliff is gone.

…t 160

Add 160 to the searchable KV tile widths and promote it in the sm_103 dense
policy, together with the deeper staging (kv_stage 6) the wider tile leaves
room for. The choice set is deliberately constant across sequence lengths:
gating it on divisibility would make the search surface length-dependent,
and the masked trailing tile makes the width legal at any length anyway.

Dense 256K also moves off packed exp2 onto the resident softmax, which is
what the search now finds for it once the seed cliff is gone.

stack-info: PR: #3576, branch: yushangdi/stack/111
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants