103. def eval_model function for evaluating model

[Gunjan-Patel-16]

Getting problem at accuracy
`
acc += accuracy_fn(y_true=y, y_pred = y_pred.argmax(dim=1))

model_0_results = eval_model(model=model_0,
data_loader=test_dataloader,
loss_fn=loss_fn,
accuracy_fn=accuracy_fn)
model_0_results`

error: 78.45
107 """
--> 108 correct = torch.eq(y_true, y_pred).sum().item()
109 acc = (correct / len(y_pred)) * 100
110 return acc

RuntimeError: The size of tensor a (16) must match the size of tensor b (32) at non-singleton dimension 0

##Last 3 batch sizes

X shape: torch.Size([32, 1, 28, 28]) | y shape: torch.Size([32]) | y_pred shape: torch.Size([32])
X shape: torch.Size([32, 1, 28, 28]) | y shape: torch.Size([32]) | y_pred shape: torch.Size([32])
X shape: torch.Size([16, 1, 28, 28]) | y shape: torch.Size([16]) | y_pred shape: torch.Size([16])

is last one causes the issue???

[LuluW8071]

RuntimeError: The size of tensor a (16) must match the size of tensor b (32) at non-singleton dimension 0

From this error
Your X (probably image) has batch_size = 16 while your y (image label) has batch_size = 32 causing shape mismatch error at non-singleton dimension 0 which is index position for batch_size

[LuluW8071]

Also just saw

model_0_results = eval_model(model=model_0,
                        data_loader=test_dataloader,
                        loss_fn=loss_fn,
                        accuracy_fn=accuracy_fn)
model_0_results

which part of code is this
Is it the training model one?

[Gunjan-Patel-16]

No its not Training model .its just basic function with evaluates model.
And here i provided link of my colab book you can go through it.

https://colab.research.google.com/drive/1VqysgzIsbBWxaz2vUudfFP2qbCYfkPEt#scrollTo=o-AeHjysDZ1v&uniqifier=2

[LuluW8071]

Unable to get access

[LuluW8071]

Typo error:

Check in your eval_model code where it should have been y_preds instead of y_pred since you have defined y_preds = model(X)

acc += accuracy_fn(y_true=y,
                         y_pred = y_pred.argmax(dim=1))

Correct way

def eval_model(model:torch.nn.Module,
              data_loader:torch.utils.data.DataLoader,
              loss_fn:torch.nn.Module,
              accuracy_fn = accuracy_fn,
              device = device):
  """Returns a dictionary containing the result of model predicting on data_loader"""
  loss,acc =0,0
  model.eval()
  with torch.inference_mode():
    for X,y in data_loader:
      
      # Make prediction
      y_preds = model(X)
      # Accumulate the loss and acc values per batch
      loss += loss_fn(y_preds,y)
      # print(f" X shape: {X.shape} | y shape: {y.shape} | y_pred shape: {y_preds.argmax(dim=1).shape}")
      acc += accuracy_fn(y_true=y,
                         y_pred = y_preds.argmax(dim=1))    # <------y_preds instead of y_pred

    # Scale loss and Acc to find average loss/acc per batch
    loss /= len(data_loader)
    acc /= len(data_loader)

  return {"Model_Name ": model.__class__.__name__,# only works when model was created with class
          "Model_loss ": loss.item(),
          "Model_Acc " : acc}

There is also same typo error in function eval_model_2 replace acc += accuracy_fn(y_true=y, y_pred = y_preds.argmax(dim=1))

Also on cell 73 of your notebook do this to avoid device error:

This is because the rand_image_tensor you passed on 'gpu' directly makes the model to call the rand_image_tensor from cpu while its on gpu and vice-versa. Hence the device error.

Corrected code

rand_image_tensor = torch.randn(size=(1,28,28)).to(device)
check_model(rand_image_tensor.unsqueeze(dim=0).to('cpu'))

or

rand_image_tensor = torch.randn(size=(1,28,28))
check_model(rand_image_tensor.unsqueeze(dim=0)

Cell 97 Use eval_model_2 function instead of eval_model

model_2_result = eval_model_2(
    model = model_2,
    data_loader = test_dataloader,
    loss_fn = loss_fn,
    accuracy_fn = accuracy_fn,
    device = device
)

Cell Block 112 Check you eval_model_2 code

You passed Model_Name and Model_Acc as Dictionary causing another typo error when plotting since there needs to be space in compare_results.set_index("Model_Name ")["Model_Acc "].plot(kind = "barh")

# Visulize our model results
compare_results.set_index("Model_Name ")["Model_Acc "].plot(kind = "barh")
plt.xlabel("Accuracy (%)")
plt.ylabel("model")

Close the discussion as answered if the error has been resolved :)

[Gunjan-Patel-16]

It works perfectly, thank you for helping me out.
I'd love to connect you for more learning.