Merge pull request #1378 from suiguoxin/quniform-tuners

fix quniform 'selected value out of bound' issue & unify quniform/qloguniform(ref to ConfigSpace)  rule for all the tuners.

docs/en_US/Tutorial/AnnotationSpec.md

@@ -41,11 +41,11 @@ There are 10 types to express your search space as follows:
 * `@nni.variable(nni.uniform(low, high),name=variable)`
   Which means the variable value is a value uniformly between low and high.
 * `@nni.variable(nni.quniform(low, high, q),name=variable)`
-  Which means the variable value is a value like round(uniform(low, high) / q) * q
+  Which means the variable value is a value like clip(round(uniform(low, high) / q) * q, low, high), where the clip operation is used to constraint the generated value in the bound.
 * `@nni.variable(nni.loguniform(low, high),name=variable)`
   Which means the variable value is a value drawn according to exp(uniform(low, high)) so that the logarithm of the return value is uniformly distributed.
 * `@nni.variable(nni.qloguniform(low, high, q),name=variable)`
-  Which means the variable value is a value like round(exp(uniform(low, high)) / q) * q
+  Which means the variable value is a value like clip(round(loguniform(low, high) / q) * q, low, high), where the clip operation is used to constraint the generated value in the bound.
 * `@nni.variable(nni.normal(mu, sigma),name=variable)`
   Which means the variable value is a real value that's normally-distributed with mean mu and standard deviation sigma.
 * `@nni.variable(nni.qnormal(mu, sigma, q),name=variable)`

docs/en_US/Tutorial/SearchSpaceSpec.md

@@ -45,15 +45,16 @@ All types of sampling strategies and their parameter are listed here:
   * When optimizing, this variable is constrained to a two-sided interval.
 
 * {"_type":"quniform","_value":[low, high, q]}
-  * Which means the variable value is a value like round(uniform(low, high) / q) * q
+  * Which means the variable value is a value like clip(round(uniform(low, high) / q) * q, low, high), where the clip operation is used to constraint the generated value in the bound. For example, for _value specified as [0, 10, 2.5], possible values are [0, 2.5, 5.0, 7.5, 10.0]; For _value specified as [2, 10, 5], possible values are [2, 5, 10].
+
   * Suitable for a discrete value with respect to which the objective is still somewhat "smooth", but which should be bounded both above and below. If you want to uniformly choose integer from a range [low, high], you can write `_value` like this: `[low, high, 1]`.
 
 * {"_type":"loguniform","_value":[low, high]}
   * Which means the variable value is a value drawn from a range [low, high] according to a loguniform distribution like exp(uniform(log(low), log(high))), so that the logarithm of the return value is uniformly distributed.
   * When optimizing, this variable is constrained to be positive.
 
 * {"_type":"qloguniform","_value":[low, high, q]}
-  * Which means the variable value is a value like round(loguniform(low, high)) / q) * q
+  * Which means the variable value is a value like clip(round(loguniform(low, high)) / q) * q, low, high), where the clip operation is used to constraint the generated value in the bound.
   * Suitable for a discrete variable with respect to which the objective is "smooth" and gets smoother with the size of the value, but which should be bounded both above and below.
 
 * {"_type":"normal","_value":[mu, sigma]}

src/sdk/pynni/nni/gridsearch_tuner/gridsearch_tuner.py

@@ -97,9 +97,8 @@ def json2parameter(self, ss_spec):
 
     def _parse_quniform(self, param_value):
         '''parse type of quniform parameter and return a list'''
-        low, high, interval = param_value[0], param_value[1], param_value[2]
-        count = int(np.floor((high - low) / interval)) + 1
-        return [low + interval * i for i in range(count)]
+        low, high, q = param_value[0], param_value[1], param_value[2]
+        return np.clip(np.arange(np.round(low/q), np.round(high/q)+1) * q, low, high)
 
     def _parse_randint(self, param_value):
         '''parse type of randint parameter and return a list'''

src/sdk/pynni/nni/hyperopt_tuner/hyperopt_tuner.py

@@ -91,7 +91,10 @@ def json2parameter(in_x, parameter, name=NodeType.ROOT):
                                    name=name + '[%d]' % _index)
                 }
             else:
-                out_y = parameter[name]
+                if _type in ['quniform', 'qloguniform']:
+                    out_y = np.clip(parameter[name], in_x[NodeType.VALUE][0], in_x[NodeType.VALUE][1])
+                else:
+                    out_y = parameter[name]
         else:
             out_y = dict()
             for key in in_x.keys():

src/sdk/pynni/nni/parameter_expressions.py

@@ -57,7 +57,7 @@ def quniform(low, high, q, random_state):
     q: sample step
     random_state: an object of numpy.random.RandomState
     '''
-    return np.round(uniform(low, high, random_state) / q) * q
+    return np.clip(np.round(uniform(low, high, random_state) / q) * q, low, high)
 
 
 def loguniform(low, high, random_state):
@@ -77,7 +77,7 @@ def qloguniform(low, high, q, random_state):
     q: sample step
     random_state: an object of numpy.random.RandomState
     '''
-    return np.round(loguniform(low, high, random_state) / q) * q
+    return np.clip(np.round(loguniform(low, high, random_state) / q) * q, low, high)
 
 
 def normal(mu, sigma, random_state):

src/sdk/pynni/nni/smartparam.py

@@ -57,14 +57,14 @@ def uniform(low, high, name=None):
 
     def quniform(low, high, q, name=None):
         assert high > low, 'Upper bound must be larger than lower bound'
-        return round(random.uniform(low, high) / q) * q
+        return np.clip(round(random.uniform(low, high) / q) * q, low, high)
 
     def loguniform(low, high, name=None):
         assert low > 0, 'Lower bound must be positive'
         return np.exp(random.uniform(np.log(low), np.log(high)))
 
     def qloguniform(low, high, q, name=None):
-        return round(loguniform(low, high) / q) * q
+        return np.clip(round(loguniform(low, high) / q) * q, low, high)
 
     def normal(mu, sigma, name=None):
         return random.gauss(mu, sigma)