【paddle.fleet】fleet add _get_applied_meta_list and _get_applied_graph_list

[wangxicoding]

PR types

Others

PR changes

APIs

Describe

fleet add _get_applied_meta_list and _get_applied_graph_list interface, which can get applied meta optimizer list and graph optimizer meta list. Fix auto amp_configs decr_ratio.
e.g.

import paddle
from paddle import fluid
import os
import paddle.distributed.fleet as fleet
paddle.enable_static()

fleet.init(is_collective=True)

input_x = paddle.fluid.layers.data(name="x", shape=[32], dtype='float32')
input_y = paddle.fluid.layers.data(name="y", shape=[1], dtype='int64')

fc_1 = paddle.fluid.layers.fc(input=input_x, size=64, act='tanh')
fc_2 = paddle.fluid.layers.fc(input=fc_1, size=256, act='tanh')
prediction = paddle.fluid.layers.fc(input=[fc_2], size=2, act='softmax')
cost = paddle.fluid.layers.cross_entropy(input=prediction, label=input_y)
avg_cost = paddle.fluid.layers.mean(x=cost)

strategy = paddle.distributed.fleet.DistributedStrategy()
strategy.amp = True
strategy.recompute = True
strategy.recompute_configs = {"checkpoints": ["fc_0.tmp_2", "fc_1.tmp_2"]}
strategy.lars = True

optimizer = paddle.fluid.optimizer.Momentum(learning_rate=0.01, momentum=0.9)
optimizer = fleet.distributed_optimizer(optimizer, strategy=strategy)
optimizer.minimize(avg_cost)

applied_meta_list = fleet._get_applied_meta_list()
print("applied_meta_list: {}".format(applied_meta_list))

We will get applied_meta_list: ['AMPOptimizer', 'RecomputeOptimizer', 'LarsOptimizer']. Which mean meta optimizer path is amp-->recompute-->lars

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[guru4elephant]

LGTM