Does my PCIe degradation affects my inference speed? I have already deployed PowerInfer successfully, but encountered a PCIe degradation problem.

[victayria77]

My PCIe is degraded from version 4.0 to 2.0, with a peak H2D speed of 10-15GB/s as my testing script shown. I have trouble reaching the 64GB/s standard with an Nvidia RTX 3090. Is this problem common? Does anybody know where to find a 64GB/s PCIe supported GPU 3090/4090...:)

[victayria77]

I have already deployed PowerInfer successfully, but encountered a PCIe degradation problem.

[SorryMaker2022]

Can this be a hardware-related problem?

For example, if you have your GPU installed in the wrong slot of your motherboard (check your motherboard's product manual for detailed info) or there are conflicts in your PCIe devices (usually when 4-5 M.2 SSDs or multiple PCIe x16 devices are installed, some of the PCIe slots will be running in x8 or x4 mode). The speed of PCIe 4.0 x8 or 3.0 x16 is 15.8 GBps.

Also, modern GPUs have dynamic PCIe lanes to save energy. GPUs dynamically degrade their PCIe protocol version / lanes when there is not so much data transfer payload. You can monitor the real-time PCIe protocol with nvidia-smi -q while running your test to see what PCIe protocol and how many lanes did your GPU utilize.

If you want 64GBps link speed I'm afraid you'll need the PCIe 5.0 x16, whose support begins at the Nvidia 50 series.

This is an interesting problem. I am curious about its cause :D