How to evaluate multiple test sets

[amritap-ef]

Hi, what is the best way to evaluate multiple test sets and get the individual accuracies?

In my data module, the test data loader is as follows, where self.test_datasets is a list of Datasets:

    def test_dataloader(self):
        return [DataLoader(ds, num_workers=self.n_workers, batch_size=self.batch_size, shuffle=False) for ds in self.test_datasets]

I have the following test_step and test_epoch_end:

def test_step(self, batch, batch_idx, dataloader_index=0):
        x, y = batch
        y_hat = self.network(x)
        loss_accuracy = self.get_loss_accuracy(y_hat, y)
        # torch.cuda.empty_cache()
        return loss_accuracy

def test_epoch_end(self, outputs):
        for i, dataloader_outputs in enumerate(outputs):
            avg_loss = torch.stack([x['loss'] for x in dataloader_outputs]).mean()
            avg_acc = sum([x['acc'] for x in dataloader_outputs]) / len(outputs)
            self.log(f'test_{i}_loss', avg_loss)
            self.log(f'test_{i}_acc', avg_acc)

But this prints out results as follows:

DATALOADER:0 TEST RESULTS

{'test_0_acc': 0.3942,

'test_0_loss': 1.900824069976807,

'test_1_acc': 0.5468,

'test_1_loss': 1.890951633453369,

'test_2_acc': 0.4979,

'test_2_loss': 1.909128665924072,

--------------------------------------------------------------------------------

DATALOADER:1 TEST RESULTS

{'test_0_acc':0.3942,

'test_0_loss': 1.900824069976807,

'test_1_acc': 0.5468,

'test_1_loss': 1.890951633453369,

'test_2_acc': 0.4979,

'test_2_loss': 1.909128665924072
}
.
.
.

Is there a good way to get the output to just print e.g. DataLoader 0 and test_0_loss_, test_0_acc, Dataloader 1 and test_1_loss, test_1_acc, etc?