Colab GPU PRO (A100) runs way slower than local CPU

[OrangeAoo]

I subscribed the GPU Pro version, which enables me to use A100 GPU. But when I was running my NN, it is way slower than running it on local vscode with a CPU!!! (My PC chip is Mac M3 pro, with 12 core GPU ).

There is no error reported and my code can run smoothly in colab. Yet the only problem is about the speed... And we can see clearly that the memory is NOT fully utilized AT ALL!!!

I still wonder if it's due to the inappropriately generated datasets (i/o issue), which might take a long time.

[LuluW8071]

Try using more complex architecture and larger batch size. A smaller batch size and less complex architecture generally result in slower speeds during training a NN. Although the GPU can also affect speed, in this case, the A100 GPU from 2020 should not be the cause. Therefore, the smaller batch size and less complex architecture are likely the reasons for the slower speeds.

[OrangeAoo]

Thank you for replying!!! I'll try escalate my sample size.

Btw, I thought about another possibility that GPU might not work well in my case. Since my code is aiming at solving the coupled PDEs, where NN models are highly coupled and intertwined, it would be impossible to parallel the computation, which is the basic trick of GPU.

Is that true?

[LuluW8071]

Yes, it's possible that GPU acceleration might not provide significant speed improvements for coupled PDEs with highly intertwined NN models. Though GPUs excel in parallelizing independent computations, but if ur calculations are tightly coupled or require frequent synchronization, GPU acceleration may not be as effective.

Reading some related research paper to ur work might provide some solutions.

[OrangeAoo]

Gotcha!!! Thank you!