Skip to content

Commit 068c3a0

Browse files
[Bugfix] Add verification for quant_action.choices to avoid TypeError (#1046)
### What this PR does / why we need it? When I run vllm-ascend, I get this error msg: ```bash Traceback (most recent call last): File "/home/sss/software/miniconda3/envs/vllm-v1/bin/vllm", line 8, in <module> sys.exit(main()) File "/home/sss/github/vllm-project/vllm/vllm/entrypoints/cli/main.py", line 50, in main cmd.subparser_init(subparsers).set_defaults( File "/home/sss/github/vllm-project/vllm/vllm/entrypoints/cli/serve.py", line 101, in subparser_init serve_parser = make_arg_parser(serve_parser) File "/home/sss/github/vllm-project/vllm/vllm/entrypoints/openai/cli_args.py", line 254, in make_arg_parser parser = AsyncEngineArgs.add_cli_args(parser) File "/home/sss/github/vllm-project/vllm/vllm/engine/arg_utils.py", line 1582, in add_cli_args current_platform.pre_register_and_update(parser) File "/home/sss/github/vllm-project/vllm-ascend/vllm_ascend/platform.py", line 80, in pre_register_and_update if ASCEND_QUATIZATION_METHOD not in quant_action.choices: TypeError: argument of type 'NoneType' is not iterable [ERROR] 2025-06-03-02:53:42 (PID:6005, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception ``` This is because the `choices` attribute in `quant_action` can be `None` and we don't check it. ```bash # quant_action _StoreAction(option_strings=['--quantization', '-q'], dest='quantization', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=False, help='Method used to quantize the weights. If `None`, we first check the\n`quantization_config` attribute in the model config file. If that is\n`None`, we assume the model weights are not quantized and use `dtype` to\ndetermine the data type of the weights.', metavar=None) ``` Thus, I have added check for the `choices` to handle the scenario of `choices=None`. ### Does this PR introduce _any_ user-facing change? yes, vllm server with ascend quantization works now. ### How was this patch tested? by `vllm server --quantization ascend` command. Related: vllm-project/vllm#19004 Signed-off-by: shen-shanshan <[email protected]> Co-authored-by: wangxiyuan <[email protected]>
1 parent 9386057 commit 068c3a0

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm_ascend/platform.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,8 @@ def pre_register_and_update(cls,
7575
# and the user can enable quantization using "vllm serve --quantization ascend".
7676
if parser is not None:
7777
quant_action = parser._option_string_actions.get('--quantization')
78-
if quant_action and hasattr(quant_action, 'choices'):
78+
if quant_action and hasattr(quant_action,
79+
'choices') and quant_action.choices:
7980
if ASCEND_QUATIZATION_METHOD not in quant_action.choices:
8081
quant_action.choices.append(ASCEND_QUATIZATION_METHOD)
8182

0 commit comments

Comments
 (0)