02_pytorch_classification Excersizes

[dwang4002]

Build a training loop for the model

Loop over data

epochs = 1000
for epoch in range(epochs):

Training

1. Forward pass

y_logits = spiral_model(X_train)
y_pred_probs = torch.softmax(y_logits, dim=1)
y_preds = torch.argmax(y_pred_probs)

2. Calculate the loss

loss = loss_fn(y_logits, y_train)
acc = accuracy_fn(y_preds, y_train)

3. Optimizer zero grad

optimizer.zero_grad()

4. Loss backward

loss.backward()

5. Optimizer step

optimizer.step()

Testing

spiral_model.eval()
with torch.inference_mode():

# 1. Forward pass
test_logits = spiral_model(X_test)
test_pred_probs = torch.softmax(test_logits,dim=1)
test_preds = torch.argmax(test_pred_probs)
  # 2. Caculate loss and acc
test_loss = loss_fn(test_logits, y_test)
test_acc = accuracy_fn(test_preds, y_test)
# Print out what's happening every 100 epochs
if epoch % 100 == 0:
  print(f"Epoch {epoch} | Loss: {loss:.4f} | Acc: {acc:.2f} | Test Loss: {test_loss:.4f} | Test Acc: {test_acc:.2f}")

---

IndexError Traceback (most recent call last)
in <cell line: 5>()
13 y_preds = torch.argmax(y_pred_probs)
14 # 2. Calculate the loss
---> 15 loss = loss_fn(y_logits, y_train)
16 acc = accuracy_fn(y_preds, y_train)
17

3 frames
/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)
3057 if size_average is not None or reduce is not None:
3058 reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 3059 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
3060
3061

IndexError: Target 2 is out of bounds.

I am using CrossEntropyLoss

[garvpatidar04]

From above what I can see is that you are using y_logits and y_train to find the loss and if I am not wrong y_logits is the model output which might be in the [[2.0, -1.0, 0.5],[-0.5, 1.0, 3.0], [1.0, 2.0, -1.0]] form if we consider 3 class for example and y_train would be like this [0,1,0].

Suppose your y_train contains class indices ranging from 0 to num_classes - 1, and your y_logits are the raw scores output by your model. In that case, the error you're encountering might be because your model's output dimension and the range of class indices are not aligned, I may be wrong but if you share the shapes and also the y_train and y_logits themself, I might be able to help you, because I am also learning PyTorch. Hence, I am new, so I can only come to this conclusion for now seeing your code.

[garvpatidar04]

If the above is not the answer then try this loss_fn(y_logits.squeeze(), y_train) it might be a problem regarding dimension because y_logits will be like this [[-0.1430],[-0.0432],[-0.2669],[-0.0772],[-0.1331]] and y_train will be like this [0., 1., 0., 1., 0.].