Skip to content

Commit bb232cc

Browse files
mcr229facebook-github-bot
authored andcommitted
Don't Decompose Hardswish
Summary: Investigating MV3, I noticed that hardswish was getting decomposed into many little ops. This become annoying because it injected unnecessary transposes, and also wasn't being quantized. I didn't realize that this was being decomposed. After doing some investigating, it looks like this can greatly improve our MV3 Performance for both Quantized and FP32 models. Some benchmarks here. | | Before Hardwish Decomp | After Hardswish Decomp | Latency Reduction | |----------------|------------------------|------------------------|-------------------| | Macbook (FP32) | [13.3685]((https://www.internalfb.com/phabricator/paste/view/P1859573931)) | [8.451](https://www.internalfb.com/phabricator/paste/view/P1859573328) |36% | | Macbook (QS8) | [16.0361](https://www.internalfb.com/phabricator/paste/view/P1859609658) | [4.914](https://www.internalfb.com/phabricator/paste/view/P1859610252) |69% | | S24 (FP32) | [56.885](https://www.internalfb.com/intern/paste/P1859603500) | [41.9638](https://www.internalfb.com/intern/paste/P1859603738) |26% | | S24 (QS8) | [56.1718](https://www.internalfb.com/intern/paste/P1859615896) | [40.2096](https://www.internalfb.com/intern/paste/P1859615683/) |40% | Reviewed By: cccclai Differential Revision: D77765129
1 parent aec1322 commit bb232cc

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

backends/xnnpack/partition/config/generic_node_configs.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -399,6 +399,9 @@ class HardswishConfig(GenericNodePartitionerConfig):
399399

400400
def supported_precision_types(self) -> List[ConfigPrecisionType]:
401401
return [ConfigPrecisionType.FP32]
402+
403+
def get_original_aten(self) -> Optional[torch._ops.OpOverload]:
404+
return torch.ops.aten.hardswish.default
402405

403406

404407
class LeakyReLUConfig(GenericNodePartitionerConfig):

0 commit comments

Comments
 (0)