Skip to content

[QNN-EP] Add Support for CumSum in QNN EP #24820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[QNN-EP] Add Support for CumSum in QNN EP #24820

wants to merge 4 commits into from

Conversation

Akupadhye
Copy link

  • Registered CumSum op in QNN EP
  • Added unit test to verify accuracy and assignment of op to QNN EP

Description

Added support for CumSum in QNN EP

Motivation and Context

There is no support for CumSum op in QNN EP

@Akupadhye
[QNN-EP] Add Support for CumSum in QNN EP
8b24fb3 
- Registered CumSum op in QNN EP
- Added unit test to verify accuracy and assignment of op to QNN EP

Description
- Added support for CumSum in QNN EP

Motivation and Context
- There is no support for CumSum op in QNN EP
@Akupadhye
Copy link
Author

@microsoft-github-policy-service agree company="Qualcomm"

@edgchen1 edgchen1 added the ep:QNN issues related to QNN exeution provider label May 20, 2025
@HectorSVC
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@HectorSVC
Copy link
Contributor

Please check the doc to make sure there are no steps missing: https://onnxruntime.ai/docs/execution-providers/QNN-ExecutionProvider.html#add-new-operator-support-in-qnn-ep
For example, make sure the DQ->CumSum->Q is treat as a single node unit, so it's running as quantized CumSum instead of FP32 in FP16 mode on HTP. Another recent example is: #24659

Akupadhye added 2 commits May 28, 2025 09:11
@Akupadhye
[QNN-EP] Add CumSum QDQ node selector
6b6c0b3 
Description
- Added CumSum QDQ node selector

Motivation and Context
- DQ->CumSum->Q is not recognized as a single qdq node unit. The CumSum
  node would run in FP16 precision in QNN HTP. Hence added the CumSum
  node unit selector to recognize it as a quantized CumSum node.
@Akupadhye
Merge branch 'main' into dev/Akupadhye/cumsum_op_support
54ad425
@Akupadhye
Copy link
Author

verified that DQ->CumSum->Q is now treated as a single node unit.

2025-05-23 11:30:00.4123831 [V:onnxruntime:qdq_model_logger, qnn_model_wrapper.cc:256 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] Qnn_OpConfig node name: node_token_7 package_name: qti.aisw QNN_op_type: CumulativeSum num_of_inputs: 1 num_of_outputs: 1 num_of_params: 3
node_inputs:
name=node id=0 version=1 type=QNN_TENSOR_TYPE_NATIVE dataFormat=0 dataType=QNN_DATATYPE_UFIXED_POINT_8 rank=2 dimensions=(3 2 ) memType=QNN_TENSORMEMTYPE_RAW quantizeParams: encodingDefinition=QNN_DEFINITION_DEFINED quantizationEncoding=QNN_QUANTIZATION_ENCODING_SCALE_OFFSET
scale=0.0282353 offset=0
node_outputs:
name=node_token_8 id=0 version=1 type=QNN_TENSOR_TYPE_NATIVE dataFormat=0 dataType=QNN_DATATYPE_UFIXED_POINT_8 rank=2 dimensions=(3 2 ) memType=QNN_TENSORMEMTYPE_RAW quantizeParams: encodingDefinition=QNN_DEFINITION_DEFINED
quantizationEncoding=QNN_QUANTIZATION_ENCODING_SCALE_OFFSET scale=0.0447059 offset=0
node_params:
type=QNN_PARAMTYPE_SCALAR name=axis value=0
type=QNN_PARAMTYPE_SCALAR name=exclusive value=0
type=QNN_PARAMTYPE_SCALAR name=reverse value=0

@HectorSVC
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

HectorSVC
HectorSVC previously approved these changes May 28, 2025
View reviewed changes
Copy link
Contributor

@HectorSVC HectorSVC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HectorSVC HectorSVC HectorSVC left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
ep:QNN issues related to QNN exeution provider
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Akupadhye @HectorSVC @edgchen1