mx.sym.constant

[jmacglashan]

For symbol constructions, there is var, zeros, and ones. Can we also get a constant one that receives a list/numpy/mx.ndarray and makes a constant symbol out of it? Working it in via variables is a bit cumbersome, particularly because the Constant initialization doesn't work for any variable with a name not ending with the string 'weight'.

[piiswrong]

yeah that's something I wanted to do but didn't have time for.
Basically you serialize an ndarray into string and pass it as argument to an op. The op then decode it and copy it to output everytime.
would be great if someone can do this

[yuruofeifei]

I'm interested in this. @piiswrong
Can we just create a const op?

[Atomu2014]

my friend has implemented a simple version of constant initializer @panzheyi

[panzheyi]

I implemented a possible solution.
A constant can be regarded as a variable which doesn't update gradient.
So you can firstly create a variable, and then block its gradient.
Here's my code:

import mxnet as mx
import numpy as np

@mx.init.register
class MyConstant(mx.init.Initializer):
    def __init__(self, value):
        super(MyConstant, self).__init__(value=value)
        self.value = value

    def _init_weight(self, _, arr):
        arr[:] = mx.nd.array(self.value)

batch_size = 10

const_arr = np.ones((5, 5)).tolist()
a = mx.sym.Variable('a', shape = (5, 5), init = MyConstant(value = const_arr))
a = mx.sym.BlockGrad(a) # now variable a is a constant

data = mx.sym.Variable('data')
loss = mx.sym.MakeLoss(mx.sym.broadcast_add(a, data))


mod = mx.mod.Module(loss, data_names = ['data'], label_names = [])
mod.bind(data_shapes = [('data', (batch_size, 5, 5)),])
mod.init_params(initializer = mx.init.Uniform())
mod.init_optimizer(optimizer = 'sgd', optimizer_params = (('learning_rate', 0.005),))


a = mx.nd.ones((5, 5))
data = np.ones((1000, 5, 5))

dataiter = mx.io.NDArrayIter(data = {'data': data}, batch_size = batch_size)
dataiter.reset()
for batch_id, databatch in enumerate(dataiter):
	mod.forward_backward(databatch)
	mod.update()
	print mod.get_outputs()[0].asnumpy() 

Variable a is a constant which is initialized by a numpy array.
Note that mx.init.Initializer only accepts the parameter which is JSON serializable.
Thus I transform the numpy array to a list before calling the function.

[csherstan]

+1

[chunyang-wen]

+1 good point

[ChuckTan123]

I have been searching for this solution for hours. Finally got one working :)

[dongzhuoyao]

any official implementation?

[AustinDoolittle]

+1 This is an essential part of network creation and would be very helpful to include natively.

[matanhs]

I've looked everywhere for a proper solution, didn't like the idea of using the special constant initializer class since there should not be a difference between scalar constant to any shape tensor. Finally what worked for me is the following:
my_var = mx.symbol.Variable('myconst',shape=(),init=mx.init.Constant(value=(a).tolist())
if you also want to block gradients simply specify the variable name as constant variable when constructing the module.

[YutingZhang]

@piiswrong Is there a plan to implement this simple feature?

[pilhoon]

@piiswrong a short example will be very helpful.