LightningCLI shouldn't run fit by default

[tchaton]

🐛 Bug

To Reproduce

Currently, the LightingCLI runs fit by default which is an anti-pattern for the Lightning API and seems like magic.

Current implementation.

    cli = LightningCLI(LiftModel, MnistDataModule)
    cli.trainer.test(datamodule=cli.datamodule)
    cli.trainer.save_checkpoint("mnist_cnn.pt")

Expected implementation.

    cli = LightningCLI(LiftModel, MnistDataModule)
    cli.trainer.fit(cli.model, datamodule=cli.datamodule)
    cli.trainer.test(datamodule=cli.datamodule)
    cli.trainer.save_checkpoint("mnist_cnn.pt")

Expected behavior

Environment

• PyTorch Lightning Version (e.g., 1.3.0):
• PyTorch Version (e.g., 1.8)
• Python version:
• OS (e.g., Linux):
• CUDA/cuDNN version:
• GPU models and configuration:
• How you installed PyTorch (conda, pip, source):
• If compiling from source, the output of torch.__config__.show():
• Any other relevant information:

Additional context

[ethanwharris]

Seems related to #7226 (PR in #7508), #8251

[awaelchli]

@tchaton I think strictly this is not a bug as it is documented here right were LightningCLI is first introduced:

The instantiation of the LightningCLI class takes care of parsing command line and config file options, instantiating the classes, setting up a callback to save the config in the log directory and finally running the trainer.

I do however agree with you that this is an issue and it is also my opinion that the instantiation of LightningCLI should not run a trainer.fit.

[Borda]

just for the record, @tchaton #8251 (comment)
cc: @carmocca

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!