Is trainer.test() supposed to call LightningDataModule.prepare_data() (again)

[hogru]

I run trainer.fit(model=my_module, datamodule=my_data) with a ModelCheckpoint() as one of the callbacks. This calls LightningDataModule.prepare_data() (as expected) and LightningDataModule.setup() with stage==TrainerFn.FITTING / "fit" (as expected).

Immediately after training, I run trainer.test(datamodule=my_data, ckpt_path="best"). This calls LightningDataModule.prepare_data() again (NOT expected) and then LightningDataModule.setup() with stage==TrainerFn.TESTING / "test" (as expected).

Is this expected behavior, i.e., trainer.test() calls prepare_data() again? If yes, is there a way to avoid this (I tried to pass the LightningDataModule.test_dataloader() to trainer.test(), but this did not change the behaviour)

[Wenbintum]

Hi. Have you found a solution for this ? I am having the same problem. For my case, I only want to prepare_data i.e., downloading stuff at the training stage

[hogru]

Unfortunately no. I "solved" this by moving the project into the hugging face environment (apparently not only for this reason). Still would be interested in learning how to do this since I probably can not / don't want to do everything in hugging face.

[Wenbintum]

I don't have a perfect solution as well. Currently, I borrow the idea from here "https://github.com/Lightning-AI/lightning/blob/6d888b5ce081277a89dc2fb9a2775b81d862fe54/src/lightning/pytorch/demos/mnist_datamodule.py#L90" where in the test stage, prepare_data() still runs but it is going to check whether It should proceed or just pass this function