Make _get_perspective_coeffs device agnostic

[ptrblck]

🐛 Describe the bug

Currently, _get_perspective_coeffs creates internal tensors on the CPU as seen here and fails in the torch.linalg.lstsq call if start/endpoints is a tensor contain data on the GPU (or another device).

The docs explain a list of list of python:ints is expected, but still tensors are allowed and do not fail.

A fix would be to create a_matrix using the device attribute of the input. An alternative would be to move the points to the host, but this would sync the code and disallow graph capture or to error out if tensors are passed.

If we want to accept tensor inputs, the tensor clone should also be fixed b_matrix = torch.tensor(startpoints, dtype=torch.float64).view(8).

Original error reported in the discussion board and reproduced using:

import torch
import torchvision.transforms.functional as TF


device = "cuda"
reference_image = torch.randn(1, 3, 224, 224, device=device)
B, C, H, W = reference_image.shape

W = 200
H = 200
# Define source points (original corners of the target image)
src_points = torch.tensor([
    [0, 0],  # Top-left
    [W - 1, 0],  # Top-right
    [W - 1, H - 1],  # Bottom-right
    [0, H - 1]  # Bottom-left
], dtype=torch.float32, device=reference_image.device)
src_points = src_points.unsqueeze(0).repeat(B, 1, 1)  # (B, 4, 2)

predicted_points = torch.tensor([
    [0, 0],  # Top-left
    [W - 10, 0],  # Top-right
    [W - 10, H - 10],  # Bottom-right
    [0, H - 10]  # Bottom-left
], dtype=torch.float32, device=reference_image.device)
predicted_points = predicted_points .unsqueeze(0).repeat(B, 1, 1)  # (B, 4, 2)

warped_images = []
for i in range(B):
    warped = TF.perspective(
        reference_image[i],
        src_points[i],
        predicted_points[i],
        interpolation=TF.InterpolationMode.BILINEAR,
        fill=0,
    )

# RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument b in method wrapper_CUDA_out_linalg_lstsq_out)```

### Versions


`0.22.0.dev20250404+cu128`

[AntoineSimoulin]

Hey @ptrblck, thanks for your time and for opening the issue! I made a proposition of changes in #9082 along with your suggestions:

• Create the a_matrix using the device attribute of the input;
• Modifiy how the b_matrix is created as well;
• Modify how the a_matrix values are filled to make sure the function was still compatible with torch script and passing tests.

Let me know what you think!

[AntoineSimoulin]

@ptrblck, we synced internally and changed our perspective on the issue. We are not sure it is necessary to amend the _get_perspective_coeffs function and merge #9082. Indeed:

• In the shared example, the tensors are explicitly created on GPU. But it seems possible to modify the code snippet to create the tensors on CPU or as [list[float]];
• The _get_perspective_coeffs function will return list[float], thus inducing a CPU/GPU communication even if the coefficients have been computed on GPU;
• This function is from the transforms.v1 and we prefer to focus on shipping upgrades to transforms.v2;
• The rests of the function's signatures from "torchvision/transforms/functional.py" are list[int] rather than torch.Tensor. to To avoid confusing users, we prefer not to introduce a discrepancy for this specific function only.

For these reasons, we are not inclined to merge #9082 as of now. Let us know what you think!