INT8 dtype tl.dot using FMA produces inaccurate results for some blocked layout input configs.
The bug was found when relaxing the dot shape requirements so that shape sizes rejected by DPAS could fallback to FMA: #7841
The problem could also currently occur if DPAS layout is rejected function-wide for dots for a function that includes an INT8 dot. The INT8 dot would then fallback to FMA path, which risks producing incorrect results. This seems like an uncommon case, but it does expose the bug, and is the case tested in the reproducer linked below.
The bug does not occur with TRITON_INTEL_ENABLE_BLOCK_IO_ALL_LAYOUTS=0, so originally thought it was related to the block-io loads for dot inputs A and B. After some further investigation, Block io loads do not seem to be the root cause. The root cause currently seems to be the matchDp4a function in IGC, where it can independently reorder elements from A and B if A was loaded by a vector load and B through scalar loads.
Bug produced on B580. Bug is not reproduced on PVC max 1100.
Environment details:
OS: Ubuntu 24.04.4 LTS
GPU: B580
IGC: 2.38.5
Pytorch: 2.14.0a0+git23b1588 commit 23b1588b32ea2d46410793347d635017fdc49c75
Triton: 3.8.0 commit ac9e26441082d190da5d304c252b26ab6a86ef90
LLVM: 24.0.0git pinned revision: 941a04e69…
Minimal reproducer: int8_fma_mixed_reproducer.py
INT8 dtype tl.dot using FMA produces inaccurate results for some blocked layout input configs.
The bug was found when relaxing the dot shape requirements so that shape sizes rejected by DPAS could fallback to FMA: #7841
The problem could also currently occur if DPAS layout is rejected function-wide for dots for a function that includes an INT8 dot. The INT8 dot would then fallback to FMA path, which risks producing incorrect results. This seems like an uncommon case, but it does expose the bug, and is the case tested in the reproducer linked below.
The bug does not occur with
TRITON_INTEL_ENABLE_BLOCK_IO_ALL_LAYOUTS=0, so originally thought it was related to the block-io loads for dot inputs A and B. After some further investigation, Block io loads do not seem to be the root cause. The root cause currently seems to be thematchDp4afunction in IGC, where it can independently reorder elements from A and B if A was loaded by a vector load and B through scalar loads.Bug produced on B580. Bug is not reproduced on PVC max 1100.
Environment details:
Minimal reproducer: int8_fma_mixed_reproducer.py