[Benchmarks] optimize fused_gemm_benchmark

[JianyuLi01]

With this change on B70, we can have ~10% uplift for large workloads (M, N, K = [220000, 8192, 512]) and ~25% uplift for small workloads (M, N, K = [256, 512, 512]).

New contributor declaration

• I am not making a trivial change, such as fixing a typo in a comment.

• I have written a PR description following these
  rules.

• I have run pre-commit run --from-ref origin/main --to-ref HEAD.

• Select one of the following.

  • I have added tests.
    • /test for lit tests
    • /unittest for C++ tests
    • /python/test for end-to-end tests
  • This PR does not need a test because FILL THIS IN.

• Select one of the following.

  • I have not added any lit tests.
  • The lit tests I have added follow these best practices,
    including the "tests should be minimal" section. (Usually running Python code
    and using the instructions it generates is not minimal.)