[Bug]: NixlConnector should not skip short do_remote_prefill requests in connector metadata #18591
Open
1 task done
Labels
bug
Something isn't working
Uh oh!
There was an error while loading. Please reload this page.
Your current environment
The output of
python collect_env.py
🐛 Describe the bug
pytest -v -x v1/kv_connector/unit/test_nixl_connector.py::test_prompt_less_than_block_size
was failed (#18490). #18429 wanted to fix it by skipping adding this request into the connector's metadata.But the failure reason is the unit test has not been updated to due the changes in NixlConnectorSheduler.
The short (promot < block_size)
do_remote_prefill
request should be copied into the connector's metadata with an emptylocal_block_ids
, so that NixlConnectorWorker can skip (async) reading remote kv blocks, but still be able to send notificaiton to the prefill worker to release its remote kv blocks.Before submitting a new issue...
The text was updated successfully, but these errors were encountered: