support setattr for some specified paramter of ParameterDict

[tornadomeet]

Description

Somewhat similar with fixed_param_names in mx.mod.Module(), which can set attr of some specified paramter.

Checklist

Essentials

• Passed code style checking (make lint)
• Changes are complete (i.e. I finished coding on this PR)
• All changes have test coverage:
• Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
• Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
• Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
• Code is well-documented:
• For user-facing API changes, API doc string has been updated.
• For new C++ functions in header files, their functionalities and arguments are documented.
• For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
• To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Comments

selected_param_names may be not a good parameter name, better name is welcome.

[piiswrong]

this can be done with a simple two line loop by the user. I don't think its worth adding an argument

[tornadomeet]

@piiswrong if we only set one attr, then we need some code like this(not concise):

all_params = net.collect_params()
fixed_params = [###]
for k, v in all_params.items():
    if k in all_params.keys():
       setattr(v, 'grad_req', 'null')

but, if we need set others attrs in the same project, such as change lr_mult, wd_mult, we need this code again, of course we can write a function to do this, but in that case why not just support this function in gluon api for better experience.

[szha]

"all" can be a valid parameter name.

[tornadomeet]

agree, '"all" is bad here.

[szha]

I agree with Eric that this option probably isn't worth adding. This option is hidden and the semantic meaning isn't that straightforward, and it only saves 4 lines at best.

Another option is to add selection functionality to collect_params, such as by name regex or name list.

[tornadomeet]

@szha how about add select_params() for class Block? so that we can do like this:

net.select_params(net.collect_params(), ['conv1_weigh', 'conv1_bias'].setattr('gred_req', 'null')

if that's fine, i'll add it.

[szha]

I think selecting params is not the concern of a block, but that of the block's param dict. Since collect_params is already the interface for getting parameters, having selection/filtering functionality there feels natural.

[tornadomeet]

do you mean add regex or name-list as a parameter of collect_params()?

[szha]

Yes

[tornadomeet]

ok, got it, i'll try.