[BUG] Unable to use GPUs besides "nvidia.com/gpu" due to hardcoding

[Sovietaced]

Flyte & Flytekit version

Flyte v1.15.3

Describe the bug

We are trying to use GPU resource types other than "nvidia.com/gpu" but there appears to be hardcoding in Flyte Propeller.

Expected behavior

I would expect Flyte Propeller not to hardcode "nvidia.com/gpu" since you can configure the GPU resource type to use in other areas.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[runllm]

Flyte supports configuring the GPU resource name via the Helm values file (e.g., gpu-resource-name), allowing you to use resource types other than "nvidia.com/gpu". However, in Flyte v1.15.3, there have been reports and code evidence that "nvidia.com/gpu" is still hardcoded in some parts of FlytePropeller, which can prevent the use of alternative GPU resource names even if configured elsewhere. This is a known limitation and has been discussed in the community and addressed in later PRs and documentation updates. See the official GPU configuration docs for details and workarounds: Flyte GPU configuration documentation.

If you need to use a custom GPU resource name, ensure you set gpu-resource-name in your FlytePropeller configuration, but be aware that full support for arbitrary GPU resource names may require a newer Flyte version where the hardcoding has been fixed. Would you like more detail on the specific code paths or upgrade recommendations?

Sources:

• Flyte GPU configuration documentation
• Flyte PR: Extended Resources GPU Accelerators
• Flyte Issue: Greater flexibility with GPUs accelerated workloads

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

[Sovietaced]

More concretely, it looks like this function has some hardcoded nvidia.com/gpu which seems to be taking precedent over what we are configuring:

flyte/flytepropeller/pkg/controller/nodes/task/taskexec_context.go

Lines 194 to 210 in 6a1efa5

	func convertTaskResourcesToRequirements(taskResources v1alpha1.TaskResources) *v1.ResourceRequirements {
	return &v1.ResourceRequirements{
	Requests: v1.ResourceList{
	v1.ResourceCPU: taskResources.Requests.CPU,
	v1.ResourceMemory: taskResources.Requests.Memory,
	v1.ResourceEphemeralStorage: taskResources.Requests.EphemeralStorage,
	utils.ResourceNvidiaGPU: taskResources.Requests.GPU,
	},
	Limits: v1.ResourceList{
	v1.ResourceCPU: taskResources.Limits.CPU,
	v1.ResourceMemory: taskResources.Limits.Memory,
	v1.ResourceEphemeralStorage: taskResources.Limits.EphemeralStorage,
	utils.ResourceNvidiaGPU: taskResources.Limits.GPU,
	},
	}
	}