Multiple dataloaders as a dict problem

[RafailFridman]

Hi, as stated here I can create multiple dataloaders and return them in a dictionary, while in the training step I should do something like:

def training_step(self, batch, batch_idx):
        # access a dictionary with a batch from each DataLoader
        batch_a = batch["a"]
        batch_b = batch["b"]

But doing it like this doesn't work, and the variable batch contains just the name of the first dataloader.
It happens only in the validation sanity check, but if I turn it off. I assume it's a bug, but not sure. Can someone check this, please?

[RafailFridman]

Update: in a plain validation loop also

[tchaton]

Dear @RafailFridman,

Training and Validation / Test are handled differently with their DataLoaders.

For Training, we support Combined Loaders represented as a collection of DataLoaders.

For Validation / Test, we assume sequential evaluation, meaning the supported format is sequence of dataloaders.

To enable the same behaviour as within Training, you could do the following:

Be careful the mode is really important and it is why we handle validation dataloader sequentially.

from pytorch_lightning.trainer.supporters import CombinedLoader


class MyDataModule(DataModule):

    def validation_dataloader(self):
        dl_1 = ...
        dl_2 = ...
        # warning: `CombinedLoader`
        # mode: the mode. Supported are 'min_size' which stops if the shortest loader is exhausted and
        # 'max_size_cycle' which stops if the longest loader is exhausted and cycles through the smaller ones.
        return CombinedLoader({"dl_1": dl_1, "dl_2": dl_2}, mode=`?`)


class Model(LightningModule):

    def validation_step(self, batch, batch_idx):
        a = batch['dl_1']
        b = batch['dl_2']

Best,
T.C

[Riccorl]

Hi @tchaton

Let's say I have multiple validation sets, and I want to track the performance of my model separately during training, how can I track which dataloader is which? In validation step, I will have only the dataloader_idx that doesn't carry such information. A dictionary could help in preserving that.

The CombinedLoader can be used as a workaround, however, it has only two modes. Is it possible to use it by simply iterate over all the data without truncating nor cycling?

[tchaton]

Your datamodule could have a function to map a dataloader_idx to a name and use this while logging.

[surya-narayanan]

Hi, just curious- it says that CombinedLoader only works with mode = sequential in the training loop?

[mshooter]

in my case it seems to reset it to mode = max_size_cycle in the training loop?? I set it to min_size but it doesn't want to change..