Skip to content

[Enhance]: Add validation for expert parallelism settings #1199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions vllm_ascend/ascend_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,14 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from typing import Optional
from typing import TYPE_CHECKING, Optional

import vllm.envs as envs
from vllm.logger import logger

if TYPE_CHECKING:
from vllm.config import VllmConfig


class AscendConfig:
"""
Expand Down Expand Up @@ -112,7 +115,7 @@ def get_ascend_config():
return _ASCEND_CONFIG


def check_ascend_config(vllm_config, enforce_eager):
def check_ascend_config(vllm_config: "VllmConfig", enforce_eager):
ascend_config = get_ascend_config()

# for v0 engine
Expand Down Expand Up @@ -164,3 +167,10 @@ def check_ascend_config(vllm_config, enforce_eager):
"ACL Graph is currently experimental. Please "
"raise an issue on https://github.com/vllm-project/vllm-ascend/issues"
" if you encourage any Error")

# for expert parallelism
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this validation into check_and_update_config in platform.py?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function will be called within check_and_update_config

if vllm_config.parallel_config.enable_expert_parallel and \
ascend_config.expert_tensor_parallel_size > 1:
raise ValueError(
"Cannot set `--enable-expert-parallel` and "
"`expert_tensor_parallel_size` > 1 at the same time.")