TrainerDDPMixin.resolve_root_node_address fails if the host name contains a dash

[LoicGrobol]

🐛 Bug

When running under SLURM, if the nodes host names contains -, MASTER_ADDR omits the number part of the master host name (which of course makes everything else fail).

if MASTER_ADDR is not given, pl infers it from the node list using TrainerDDPMixin.resolve_root_node_address

To Reproduce

Steps to reproduce the behavior:

• Use pl in a SLURM cluster where nodes host names contains at least a dash (e.g. jean-zay-ia810)

The job should fail with something like this

  File "<snip>/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 834, in fit
    self.ddp_train(task, model)
  File "<snip>/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_data_parallel.py", line 353, in ddp_train
    model.init_ddp_connection(self.proc_rank, self.world_size, self.is_slurm_managing_tasks)
  File "<snip>/lib/python3.7/site-packages/pytorch_lightning/core/lightning.py", line 957, in init_ddp_connection
    torch_distrib.init_process_group(torch_backend, rank=proc_rank, world_size=world_size)
  File "<snip>/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py", line 393, in init_process_group
    store, rank, world_size = next(rendezvous_iterator)
  File "<snip>/lib/python3.7/site-packages/torch/distributed/rendezvous.py", line 172, in _env_rendezvous_handler
    store = TCPStore(master_addr, master_port, world_size, start_daemon, timeout)
ValueError: host not found: Name or service not known

In my case, looking into os.environ gives 'SLURM_NODELIST': 'jean-zay-ia[810,817-819]' and 'MASTER_ADDR': 'jean-zay-ia'.

The root cause being that

>>> resolve_root_node_address("jean-zay-ia[810,817-819]")
'jean-zay-ia'

Fixing it should not be hard, should I submit a PR?

[github-actions]

Hi! thanks for your contribution!, great first issue!