Skip to content

Commit 5f5800b

Browse files
authored
[Bugfix] Sync MRotaryEmbedding interface change to recover CI (#1399)
### What this PR does / why we need it? Sync MRotaryEmbedding interface change to recover main CI (vllm-project/vllm#19939) ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed --------- Signed-off-by: wangli <[email protected]>
1 parent 6ed3f00 commit 5f5800b

File tree

1 file changed

+17
-7
lines changed

1 file changed

+17
-7
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -274,6 +274,7 @@ def __init__(self, vllm_config: VllmConfig, device: torch.device):
274274
dtype=torch.int64,
275275
device="cpu",
276276
pin_memory=True)
277+
self.mrope_positions_np = self.mrope_positions_cpu.numpy()
277278

278279
if self.is_multimodal_model:
279280
self.inputs_embeds = torch.zeros(
@@ -793,14 +794,23 @@ def _calc_mrope_positions(self, scheduler_output: "SchedulerOutput"):
793794
dst_start = mrope_pos_ptr
794795
dst_end = mrope_pos_ptr + completion_part_len
795796

796-
self.mrope_positions_cpu[:, dst_start:dst_end] = \
797+
if vllm_version_is("0.9.1"):
798+
self.mrope_positions_cpu[:, dst_start:dst_end] = \
799+
MRotaryEmbedding.get_next_input_positions_tensor(
800+
req.mrope_position_delta,
801+
context_len=num_computed_tokens +
802+
prompt_part_len,
803+
seq_len=num_computed_tokens +
804+
prompt_part_len +
805+
completion_part_len,
806+
)
807+
else:
797808
MRotaryEmbedding.get_next_input_positions_tensor(
798-
req.mrope_position_delta,
799-
context_len=num_computed_tokens +
800-
prompt_part_len,
801-
seq_len=num_computed_tokens +
802-
prompt_part_len +
803-
completion_part_len,
809+
out=self.mrope_positions_np,
810+
out_offset=dst_start,
811+
mrope_position_delta=req.mrope_position_delta,
812+
context_len=num_computed_tokens + prompt_part_len,
813+
num_new_tokens=completion_part_len,
804814
)
805815

806816
mrope_pos_ptr += completion_part_len

0 commit comments

Comments
 (0)