[X86] Reuse unchanged incoming stack args for tail calls#208248
Conversation
|
Hello @AxXxB 👋 Thank you for submitting a Pull Request (PR) to the LLVM Project. Since this is your first PR, here are a few useful links covering our main contribution policies and review practices.
Please reply to this message to confirm that you have read these policies, especially the LLVM AI Tool Use Policy, and that any AI tool usage has been noted in the PR description. Frequently asked questionsHow do I add reviewers? This PR will be automatically labeled, and the relevant teams will be notified. For some parts of the project, reviewers may also be added automatically. You can also add reviewers manually using the Reviewers section on this page. If you cannot use that section, it is probably because you do not have write permissions for the repository. In that case, you can request a review by tagging reviewers in a comment using What if there are no comments? If you have not received any comments on your PR after a week, you can request a review by pinging the PR with a comment such as “Ping”. The common courtesy ping rate is once a week. Please remember that you are asking for volunteer time from other developers. Are any special GitHub settings required to contribute to LLVM? We only require contributors to have a public email address associated with their GitHub commits, see this section of LLVM Developer Policy for details. If you have questions, feel free to leave a comment on this PR, or ask on LLVM Discord or LLVM Discourse. Thank you, |
|
I have read the LLVM AI Tool Use Policy, Code-Review Policy and Practices, and Developer Policy. AI tool usage is noted in the PR description. |
|
@llvm/pr-subscribers-backend-x86 Author: AxXxB ChangesWhen a musttail call forwards an incoming stack argument unchanged to the Reuse the incoming stack slot when MatchingStackOffset proves that the The test covers unchanged stack arguments, a changed stack argument, and a Tested with Assisted-by: OpenAI Codex Full diff: https://github.com/llvm/llvm-project/pull/208248.diff 2 Files Affected:
diff --git a/llvm/lib/Target/X86/X86ISelLoweringCall.cpp b/llvm/lib/Target/X86/X86ISelLoweringCall.cpp
index 819e0a023c1c5..335b76eef955b 100644
--- a/llvm/lib/Target/X86/X86ISelLoweringCall.cpp
+++ b/llvm/lib/Target/X86/X86ISelLoweringCall.cpp
@@ -2084,6 +2084,28 @@ X86TargetLowering::ByValCopyKind X86TargetLowering::ByValNeedsCopyForTailCall(
return CopyViaTemp;
}
+static bool MatchingStackOffset(SDValue Arg, unsigned Offset,
+ ISD::ArgFlagsTy Flags, MachineFrameInfo &MFI,
+ const MachineRegisterInfo *MRI,
+ const X86InstrInfo *TII,
+ const CCValAssign &VA);
+
+static bool canReuseIncomingTailCallStackArg(
+ SDValue Arg, int32_t Offset, ISD::ArgFlagsTy Flags, MachineFunction &MF,
+ const X86Subtarget &Subtarget, const CCValAssign &VA) {
+ // ByVal tailcall arguments have extra temporary-copy handling above this
+ // point. Keep the first reuse pass limited to direct scalar/vector stack
+ // arguments, matching the common musttail same-signature case.
+ if (Flags.isByVal())
+ return false;
+
+ if (Offset < 0)
+ return false;
+
+ return MatchingStackOffset(Arg, unsigned(Offset), Flags, MF.getFrameInfo(),
+ &MF.getRegInfo(), Subtarget.getInstrInfo(), VA);
+}
+
SDValue
X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
SmallVectorImpl<SDValue> &InVals) const {
@@ -2532,8 +2554,14 @@ X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
if (Flags.isInAlloca() || Flags.isPreallocated())
continue;
// Create frame index.
- int32_t Offset = VA.getLocMemOffset()+FPDiff;
- uint32_t OpSize = (VA.getLocVT().getSizeInBits()+7)/8;
+ int32_t Offset = VA.getLocMemOffset() + FPDiff;
+ uint32_t OpSize = (VA.getLocVT().getSizeInBits() + 7) / 8;
+
+ if (Is64Bit && isTailCall &&
+ canReuseIncomingTailCallStackArg(Arg, Offset, Flags, MF, Subtarget,
+ VA))
+ continue;
+
FI = MF.getFrameInfo().CreateFixedObject(OpSize, Offset, true);
FIN = DAG.getFrameIndex(FI, getPointerTy(DAG.getDataLayout()));
diff --git a/llvm/test/CodeGen/X86/preserve-nonecc-tailcall-stack-args.ll b/llvm/test/CodeGen/X86/preserve-nonecc-tailcall-stack-args.ll
new file mode 100644
index 0000000000000..939efecf417c7
--- /dev/null
+++ b/llvm/test/CodeGen/X86/preserve-nonecc-tailcall-stack-args.ll
@@ -0,0 +1,51 @@
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -relocation-model=static | FileCheck %s
+
+declare preserve_nonecc void @next(i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64)
+declare preserve_nonecc void @next_with_cond(i1, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64)
+
+define preserve_nonecc void @static_tail_unchanged(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, i64 %a7, i64 %a8, i64 %a9, i64 %a10, i64 %a11, i64 %a12, i64 %a13, i64 %a14, i64 %a15, i64 %a16, i64 %a17, i64 %a18, i64 %a19, i64 %a20, i64 %a21, i64 %a22, i64 %a23, i64 %a24, i64 %a25, i64 %a26, i64 %a27, i64 %a28, i64 %a29, i64 %a30, i64 %a31, i64 %a32, i64 %a33, i64 %a34, i64 %a35) {
+; CHECK-LABEL: static_tail_unchanged:
+; CHECK: # %bb.0:
+; CHECK-NEXT: jmp next@PLT # TAILCALL
+entry:
+ musttail call preserve_nonecc void @next(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, i64 %a7, i64 %a8, i64 %a9, i64 %a10, i64 %a11, i64 %a12, i64 %a13, i64 %a14, i64 %a15, i64 %a16, i64 %a17, i64 %a18, i64 %a19, i64 %a20, i64 %a21, i64 %a22, i64 %a23, i64 %a24, i64 %a25, i64 %a26, i64 %a27, i64 %a28, i64 %a29, i64 %a30, i64 %a31, i64 %a32, i64 %a33, i64 %a34, i64 %a35)
+ ret void
+}
+
+define preserve_nonecc void @static_tail_changed(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, i64 %a7, i64 %a8, i64 %a9, i64 %a10, i64 %a11, i64 %a12, i64 %a13, i64 %a14, i64 %a15, i64 %a16, i64 %a17, i64 %a18, i64 %a19, i64 %a20, i64 %a21, i64 %a22, i64 %a23, i64 %a24, i64 %a25, i64 %a26, i64 %a27, i64 %a28, i64 %a29, i64 %a30, i64 %a31, i64 %a32, i64 %a33, i64 %a34, i64 %a35) {
+; CHECK-LABEL: static_tail_changed:
+; CHECK: # %bb.0:
+; CHECK-NEXT: movq 168(%rsp), %r10
+; CHECK-NEXT: incq %r10
+; CHECK-NEXT: movq %r10, 168(%rsp)
+; CHECK-NEXT: jmp next@PLT # TAILCALL
+entry:
+ %a32x = add i64 %a32, 1
+ musttail call preserve_nonecc void @next(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, i64 %a7, i64 %a8, i64 %a9, i64 %a10, i64 %a11, i64 %a12, i64 %a13, i64 %a14, i64 %a15, i64 %a16, i64 %a17, i64 %a18, i64 %a19, i64 %a20, i64 %a21, i64 %a22, i64 %a23, i64 %a24, i64 %a25, i64 %a26, i64 %a27, i64 %a28, i64 %a29, i64 %a30, i64 %a31, i64 %a32x, i64 %a33, i64 %a34, i64 %a35)
+ ret void
+}
+
+define preserve_nonecc void @phi_tail(i1 %cond, i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, i64 %a7, i64 %a8, i64 %a9, i64 %a10, i64 %a11, i64 %a12, i64 %a13, i64 %a14, i64 %a15, i64 %a16, i64 %a17, i64 %a18, i64 %a19, i64 %a20, i64 %a21, i64 %a22, i64 %a23, i64 %a24, i64 %a25, i64 %a26, i64 %a27, i64 %a28, i64 %a29, i64 %a30, i64 %a31, i64 %a32, i64 %a33, i64 %a34, i64 %a35) {
+; CHECK-LABEL: phi_tail:
+; CHECK: # %bb.0:
+; CHECK-NEXT: movq 176(%rsp), %r10
+; CHECK-NEXT: testb $1, %dil
+; CHECK-NEXT: je .LBB2_2
+; CHECK: # %bb.1:
+; CHECK-NEXT: incq %r10
+; CHECK: .LBB2_2:
+; CHECK-NEXT: movq %r10, 176(%rsp)
+; CHECK-NEXT: movzbl %dil, %edi
+; CHECK-NEXT: jmp next_with_cond@PLT # TAILCALL
+entry:
+ br i1 %cond, label %a, label %b
+a:
+ %a32x = add i64 %a32, 1
+ br label %join
+b:
+ br label %join
+join:
+ %a32m = phi i64 [ %a32x, %a ], [ %a32, %b ]
+ musttail call preserve_nonecc void @next_with_cond(i1 %cond, i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, i64 %a7, i64 %a8, i64 %a9, i64 %a10, i64 %a11, i64 %a12, i64 %a13, i64 %a14, i64 %a15, i64 %a16, i64 %a17, i64 %a18, i64 %a19, i64 %a20, i64 %a21, i64 %a22, i64 %a23, i64 %a24, i64 %a25, i64 %a26, i64 %a27, i64 %a28, i64 %a29, i64 %a30, i64 %a31, i64 %a32m, i64 %a33, i64 %a34, i64 %a35)
+ ret void
+}
|
You can test this locally with the following command:git-clang-format --diff origin/main HEAD --extensions cpp -- llvm/lib/Target/X86/X86ISelLoweringCall.cpp --diff_from_common_commit
View the diff from clang-format here.diff --git a/llvm/lib/Target/X86/X86ISelLoweringCall.cpp b/llvm/lib/Target/X86/X86ISelLoweringCall.cpp
index 335b76eef..d5b15253f 100644
--- a/llvm/lib/Target/X86/X86ISelLoweringCall.cpp
+++ b/llvm/lib/Target/X86/X86ISelLoweringCall.cpp
@@ -2087,12 +2087,13 @@ X86TargetLowering::ByValCopyKind X86TargetLowering::ByValNeedsCopyForTailCall(
static bool MatchingStackOffset(SDValue Arg, unsigned Offset,
ISD::ArgFlagsTy Flags, MachineFrameInfo &MFI,
const MachineRegisterInfo *MRI,
- const X86InstrInfo *TII,
- const CCValAssign &VA);
+ const X86InstrInfo *TII, const CCValAssign &VA);
-static bool canReuseIncomingTailCallStackArg(
- SDValue Arg, int32_t Offset, ISD::ArgFlagsTy Flags, MachineFunction &MF,
- const X86Subtarget &Subtarget, const CCValAssign &VA) {
+static bool canReuseIncomingTailCallStackArg(SDValue Arg, int32_t Offset,
+ ISD::ArgFlagsTy Flags,
+ MachineFunction &MF,
+ const X86Subtarget &Subtarget,
+ const CCValAssign &VA) {
// ByVal tailcall arguments have extra temporary-copy handling above this
// point. Keep the first reuse pass limited to direct scalar/vector stack
// arguments, matching the common musttail same-signature case.
|
When a musttail call forwards an incoming stack argument unchanged to the same outgoing stack offset, X86 lowering still creates a fixed object and stores the value back to the stack. In the common same-signature tail-call case this materializes unnecessary stack traffic before the tail jump. Reuse the incoming stack slot when MatchingStackOffset proves that the outgoing argument already resides at the required offset. Keep byval arguments on the existing copy path and limit the first reuse path to non-negative fixed stack offsets. Assisted-by: OpenAI Codex
74816cc to
aae152a
Compare
|
Applied clang-format changes |
|
PR branch updated to current main. |
1 participant
When a musttail call forwards an incoming stack argument unchanged to the
same outgoing stack offset, X86 lowering still creates a fixed object and
stores the value back to the stack. In the common same-signature tail-call
case this materializes unnecessary stack traffic before the tail jump.
Reuse the incoming stack slot when MatchingStackOffset proves that the
outgoing argument already resides at the required offset. Keep byval
arguments on the existing copy path and limit the first reuse path to
non-negative fixed stack offsets.
The test covers unchanged stack arguments, a changed stack argument, and a
phi case where only the modified stack argument is stored.
Tested with
ninja -C build -j8 llcandbuild/bin/llvm-lit -q llvm/test/CodeGen/X86.AI tool disclosure: Co-authored with OpenAI Codex.