Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Add portList config in PAI trainingService #1467

Merged
merged 6 commits into from
Aug 26, 2019

Conversation

SparkSnail
Copy link
Contributor

@SparkSnail SparkSnail commented Aug 14, 2019

  1. Add portList to PAI mode configurations #1461
  2. fix error link in supportTrainingService.md

@ultmaster
Copy link
Contributor

It's working now, but the process is not straightforward (at least to me). Posting an example here:

Let's say you want to launch a tensorboard in the mnist example. So the first step is to write a wrapper script of mnist.py.

pip3 install tensorboard
export TENSORBOARD_PORT=PAI_PORT_LIST_${PAI_CURRENT_TASK_ROLE_NAME}_0_tensorboard
tensorboard --logdir . --port ${!TENSORBOARD_PORT} &
python3 mnist.py

According to this discussion, you need to use indirect expansion ${!XXX} to get the port here, and therefore it's also required to use bash instead of sh in the next step.

Besides, config_pai.yml needs to specify that "it wants a port".

trial:
  command: bash launch_pai.sh
  portList:
    - label: tensorboard
      beginAt: 0
      portNumber: 1

@QuanluZhang QuanluZhang requested a review from yds05 August 16, 2019 08:13
The config schema in NNI is shown below:
```
portList:
- label: test,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove extra ','

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed.

@SparkSnail SparkSnail merged commit 704b60e into microsoft:v1.0 Aug 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants