Cast Nodes Fusion

[nenad1002]

Description

We might have a case where multiple Cast nodes in the chain cast back to the original type. This fusion will remove extra nodes.
E.g.
A ('float32') -> Cast (to='float16') -> Cast (to='int4') -> Cast (to='float32') -> Cast (to='float16') -> B
will reduce to
A ('float32') -> Cast (to='float16') -> B
All the Cast nodes throughout the path need to have one input and one output to be considered for the fusion.

Motivation and Context

Gemma3 ONNX models used to have double casting, and many new models created by the model builder might have as well. Extra Casts might reduce accuracy and increase inference time.

[github-actions]

You can commit the suggested changes from lintrunner.

[tianleiwu]

We used to have similar Cast removing logic but it causes a lot of accuracy issues. That was reversed in #17953 to be more conservative. I suggest to add an option in onnxruntime_session_options_config_keys.h. The default is off, and user can turn it on if needed.

It is easy to process model offline like

onnxruntime/onnxruntime/python/tools/transformers/onnx_model.py

Line 667 in ad7b0e3

def remove_cascaded_cast_nodes(self):

.
So another way to avoid the issue to use such post-processing in model builder.

[nenad1002]

We used to have similar Cast removing logic but it causes a lot of accuracy issues. That was reversed in #17953 to be more conservative. I suggest to add an option in onnxruntime_session_options_config_keys.h. The default is off, and user can turn it on if needed.

It is easy to process model offline like

onnxruntime/onnxruntime/python/tools/transformers/onnx_model.py

Line 667 in ad7b0e3

def remove_cascaded_cast_nodes(self):

.
So another way to avoid the issue to use such post-processing in model builder.

Ok, interesting, did not know this could cause accuracy drop. I can go either way, add a feature filter option or just rely on offline model processing. It is nice to have onnxruntime remove it for us automatically though since we often end up with multiple casts when experimenting.