[Feat] Restore and validate KleidiAI INT4 quantization path using updated quantizer API #2570
Conversation
…ated quantizer API - Switched to quantize_() with Int8DynamicActivationIntxWeightConfig - Validated the move of packed_linear_int8_dynamic_activation_intx_weight_layout.py in torchao/dtypes/uintx - Fixed handling of SYMMETRIC_NO_CLIPPING_ERR mapping type - Validated INT4 path on a 2-layer nn.Sequential model with torch.int4 weights - Compared SYMMETRIC vs SYMMETRIC_NO_CLIPPING_ERR across PerAxis and PerGroup granularities
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2570
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hi @gausah-arm! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
This PR validates the support for the INT4 quantization path using KleidiAI by checking the integration to use the new
quantize_()API andInt8DynamicActivationIntxWeightConfig. The PR addresses breakages like Symmetric_no_clipping_err caused by API refactors and ensures compatibility with the current layout.