add top_hit metric

Backports PR elastic#7302 **Commit 1:** add top_hits aggregation * Original sha: 22bab62 * Authored by Stéphane Campinas <[email protected]> on 2016-05-26T22:02:08Z **Commit 2:** support nested fields * Original sha: 8aef2b8 * Authored by Stéphane Campinas <[email protected]> on 2016-06-11T14:57:35Z **Commit 3:** added choice of the order and select by default the index pattern associated time field * Original sha: 9791ac5 * Authored by Stéphane Campinas <[email protected]> on 2016-06-14T21:08:00Z **Commit 4:** improved parameters name and changed aggregation from latest to top to better reflect what this aggregation may accomplish * Original sha: 692f41c * Authored by Stéphane Campinas <[email protected]> on 2016-06-22T20:32:12Z **Commit 5:** made variable a constant * Original sha: 39cddab * Authored by Stéphane Campinas <[email protected]> on 2016-07-01T22:42:13Z **Commit 6:** use shorter name for variable * Original sha: c6494cc * Authored by Stéphane Campinas <[email protected]> on 2016-07-27T08:50:44Z **Commit 7:** prevent to sort terms on top_hits * Original sha: b365ddc * Authored by Stéphane Campinas <[email protected]> on 2016-08-28T09:47:47Z **Commit 8:** Added the top_hits agg to line/pie/tile vis. Added support for function in prop_filter to allow more flexibility when filtering the list of values. The _term order is now the default ordering instead of _custom. * Original sha: be44e90 * Authored by Stéphane Campinas <[email protected]> on 2016-09-19T14:53:48Z **Commit 9:** simplified tests * Original sha: 43767f0 * Authored by Stéphane Campinas <[email protected]> on 2016-10-11T09:29:11Z **Commit 10:** changed as per code review * Original sha: 53469d5 * Authored by Stéphane Campinas <[email protected]> on 2016-10-11T21:49:36Z **Commit 11:** Top Hit should support all field types on the metric vis * Original sha: 20be950 * Authored by Stéphane Campinas <[email protected]> on 2016-10-17T21:18:31Z **Commit 12:** renamed metric to top hit * Original sha: 20cbb12 * Authored by Stéphane Campinas <[email protected]> on 2016-10-17T21:28:30Z **Commit 13:** check if there is any result from buckets * Original sha: b99d6c7 * Authored by Stéphane Campinas <[email protected]> on 2016-10-18T09:13:19Z **Commit 14:** support scripted fields * Original sha: f79cdc1 * Authored by Stéphane Campinas <[email protected]> on 2016-10-23T20:58:55Z **Commit 15:** Merge branch 'master' of github.com:elastic/kibana into latest-value * Original sha: 82060ce * Authored by Stéphane Campinas <[email protected]> on 2016-11-01T09:47:19Z **Commit 16:** Merge branch 'master' of github.com:elastic/kibana into latest-value * Original sha: 2577298 * Authored by Stéphane Campinas <[email protected]> on 2016-11-12T11:30:42Z **Commit 17:** - added logic for trying to get the field value from the source or from a doc_values field. - added onlyAggregatable option for a field agg param to decide whether or not to retain only aggregatable fields. * Original sha: 6a2bc01 * Authored by Stéphane Campinas <[email protected]> on 2016-11-13T12:34:10Z **Commit 18:** improved loops in getValuesAtPath method * Original sha: 991864c * Authored by Stéphane Campinas <[email protected]> on 2016-11-16T10:10:28Z **Commit 19:** Merge branch 'master' of github.com:elastic/kibana into latest-value * Original sha: bc1c90a * Authored by Stéphane Campinas <[email protected]> on 2016-11-17T23:51:13Z **Commit 20:** do not try to get the doc_values field of an IP field * Original sha: d697f11 * Authored by Stéphane Campinas <[email protected]> on 2016-11-18T00:49:34Z **Commit 21:** improved field tests on the onlyAggregatable option * Original sha: c2249e5 * Authored by Stéphane Campinas <[email protected]> on 2016-11-18T01:08:34Z **Commit 22:** use indexPattern.formatField to get all the values at a given path * Original sha: f4200e1 * Authored by Stéphane Campinas <[email protected]> on 2016-11-19T23:02:18Z **Commit 23:** do not show the analyzed warning * Original sha: 2dda53d * Authored by Stéphane Campinas <[email protected]> on 2016-11-19T23:30:59Z **Commit 24:** support to sort on scripted field * Original sha: eda9610 * Authored by Stéphane Campinas <[email protected]> on 2016-11-19T23:52:15Z **Commit 25:** corrected tests and rely on the field's formatter instead of trying to return a nice string * Original sha: b465438 * Authored by Stéphane Campinas <[email protected]> on 2016-11-20T14:40:43Z **Commit 26:** do not rely on init_default_field_props to set the doc_values property * Original sha: 19e2104 * Authored by Stéphane Campinas <[email protected]> on 2016-11-21T23:11:29Z **Commit 27:** added test for the doc_values property * Original sha: fd323f3 * Authored by Stéphane Campinas <[email protected]> on 2016-11-23T10:04:16Z **Commit 28:** Merge branch 'master' of github.com:elastic/kibana into latest-value * Original sha: cfed72e * Authored by Stéphane Campinas <[email protected]> on 2016-12-05T15:47:45Z **Commit 29:** - corrected source formatter - add option to show/hide analyzed warning of a field * Original sha: 3090182 * Authored by Stéphane Campinas <[email protected]> on 2016-12-05T22:40:14Z **Commit 30:** set the default value of the showAnalyzedWarning in fieldparam * Original sha: 0d924fc * Authored by Stéphane Campinas <[email protected]> on 2016-12-08T16:29:46Z **Commit 31:** added description of the showAnalyzedWarning option * Original sha: 7f85803 * Authored by Stéphane Campinas <[email protected]> on 2016-12-08T16:39:07Z **Commit 32:** use aggParam object to access the options of the field parameter * Original sha: ef95ceb * Authored by Stéphane Campinas <[email protected]> on 2016-12-09T10:54:51Z **Commit 33:** flattenHit: support object in array * Original sha: b64f92a * Authored by Stéphane Campinas <[email protected]> on 2016-12-10T23:36:45Z **Commit 34:** added aggregation option to support multivalued fields and top_hits aggregation with a size different than 1 * Original sha: e18a30d * Authored by Stéphane Campinas <[email protected]> on 2016-12-17T13:49:17Z **Commit 35:** disable concatenate aggregation type for visualizations other than table and metric * Original sha: 2a2fa67 * Authored by Stéphane Campinas <[email protected]> on 2016-12-17T15:00:08Z **Commit 36:** corrected display of arrays in the metric vis * Original sha: eb70e84 * Authored by Stéphane Campinas <[email protected]> on 2016-12-17T15:27:36Z **Commit 37:** removed condition on the IP field type when retrieving doc values * Original sha: d19ba56 * Authored by Stéphane Campinas <[email protected]> on 2016-12-17T21:30:47Z **Commit 38:** - do not set a default value for the aggregate with property since it is * Original sha: 66d3b31 * Authored by Stéphane Campinas <[email protected]> on 2016-12-21T23:04:12Z **Commit 39:** filter out incompatble aggregate options and auto select the option if there is only one * Original sha: 1bd4727 * Authored by Stéphane Campinas <[email protected]> on 2016-12-28T17:14:21Z **Commit 40:** simplified condition * Original sha: 5babf9a * Authored by Stéphane Campinas <[email protected]> on 2016-12-28T23:49:46Z **Commit 41:** lint * Original sha: 7372fce * Authored by Stéphane Campinas <[email protected]> on 2016-12-30T19:43:08Z **Commit 42:** fix min/max/sum/average aggregate of elements having only null/undefined values * Original sha: 0791be7 * Authored by Stéphane Campinas <[email protected]> on 2017-01-03T10:40:16Z **Commit 43:** added top_hits metric to heatmap visualization * Original sha: c967a23 * Authored by Stéphane Campinas <[email protected]> on 2017-01-03T11:16:56Z
scampi · Jan 11, 2017 · e415c68 · e415c68
1 parent ab6a18c
commit e415c68
Show file tree

Hide file tree

Showing 31 changed files with 1,043 additions and 285 deletions.
diff --git a/src/core_plugins/kbn_vislib_vis_types/public/heatmap.js b/src/core_plugins/kbn_vislib_vis_types/public/heatmap.js
@@ -65,7 +65,7 @@ export default function HeatmapVisType(Private) {
         title: 'Value',
         min: 1,
         max: 1,
-        aggFilter: ['count', 'avg', 'median', 'sum', 'min', 'max', 'cardinality', 'std_dev'],
+        aggFilter: ['count', 'avg', 'median', 'sum', 'min', 'max', 'cardinality', 'std_dev', 'top_hits'],
         defaults: [
           { schema: 'metric', type: 'count' }
         ]

diff --git a/src/core_plugins/kbn_vislib_vis_types/public/line.js b/src/core_plugins/kbn_vislib_vis_types/public/line.js
@@ -70,7 +70,7 @@ export default function HistogramVisType(Private) {
         title: 'Dot Size',
         min: 0,
         max: 1,
-        aggFilter: ['count', 'avg', 'sum', 'min', 'max', 'cardinality']
+        aggFilter: ['count', 'avg', 'sum', 'min', 'max', 'cardinality', 'top_hits']
       },
       {
         group: 'buckets',

diff --git a/src/core_plugins/kbn_vislib_vis_types/public/pie.js b/src/core_plugins/kbn_vislib_vis_types/public/pie.js
@@ -44,7 +44,7 @@ export default function HistogramVisType(Private) {
         title: 'Slice Size',
         min: 1,
         max: 1,
-        aggFilter: ['sum', 'count', 'cardinality'],
+        aggFilter: ['sum', 'count', 'cardinality', 'top_hits'],
         defaults: [
           { schema: 'metric', type: 'count' }
         ]

diff --git a/src/core_plugins/kbn_vislib_vis_types/public/tile_map.js b/src/core_plugins/kbn_vislib_vis_types/public/tile_map.js
@@ -89,7 +89,7 @@ export default function TileMapVisType(Private, getAppState, courier, config) {
         title: 'Value',
         min: 1,
         max: 1,
-        aggFilter: ['count', 'avg', 'sum', 'min', 'max', 'cardinality'],
+        aggFilter: ['count', 'avg', 'sum', 'min', 'max', 'cardinality', 'top_hits'],
         defaults: [
           { schema: 'metric', type: 'count' }
         ]

diff --git a/src/core_plugins/kibana/public/visualize/editor/agg_params.js b/src/core_plugins/kibana/public/visualize/editor/agg_params.js
@@ -82,11 +82,18 @@ uiModules
         // build collection of agg params html
         type.params.forEach(function (param, i) {
           let aggParam;
+          let fields;
+          // if field param exists, compute allowed fields
+          if (param.name === 'field') {
+            fields = $aggParamEditorsScope.indexedFields;
+          } else if (param.type === 'field') {
+            fields = $aggParamEditorsScope[`${param.name}Options`] = getIndexedFields(param);
+          }
 
-          if ($aggParamEditorsScope.indexedFields) {
-            const hasIndexedFields = $aggParamEditorsScope.indexedFields.length > 0;
+          if (fields) {
+            const hasIndexedFields = fields.length > 0;
             const isExtraParam = i > 0;
-            if (!hasIndexedFields && isExtraParam) { // don't draw the rest of the options if their are no indexed fields.
+            if (!hasIndexedFields && isExtraParam) { // don't draw the rest of the options if there are no indexed fields.
               return;
             }
           }
@@ -133,6 +140,31 @@ uiModules
         .append(param.editor)
         .get(0);
       }
+
+      function getIndexedFields(param) {
+        let fields = _.filter($scope.agg.vis.indexPattern.fields.raw, 'aggregatable');
+        const fieldTypes = param.filterFieldTypes;
+
+        if (fieldTypes) {
+          const filter = _.isFunction(fieldTypes) ? fieldTypes.bind(this, $scope.agg.vis) : fieldTypes;
+          fields = $filter('fieldType')(fields, filter);
+          fields = $filter('orderBy')(fields, ['type', 'name']);
+        }
+
+        return new IndexedArray({
+
+          /**
+           * @type {Array}
+           */
+          index: ['name'],
+
+          /**
+           * [group description]
+           * @type {Array}
+           */
+          initialSet: fields
+        });
+      }
     }
   };
 });
diff --git a/src/core_plugins/kibana/server/lib/__tests__/init_default_field_props.js b/src/core_plugins/kibana/server/lib/__tests__/init_default_field_props.js
diff --git a/src/core_plugins/kibana/server/lib/init_default_field_props.js b/src/core_plugins/kibana/server/lib/init_default_field_props.js
diff --git a/src/core_plugins/kibana/server/lib/mapping_overrides.js b/src/core_plugins/kibana/server/lib/mapping_overrides.js
diff --git a/src/core_plugins/metric_vis/public/__tests__/metric_vis_controller.js b/src/core_plugins/metric_vis/public/__tests__/metric_vis_controller.js
@@ -20,15 +20,8 @@ describe('metric vis', function () {
   it('should set the metric label and value', function () {
     $scope.processTableGroups({
       tables: [{
-        columns: [{title: 'Count'}],
-        rows: [[4301021]],
-        aggConfig: function () {
-          return {
-            fieldFormatter: function () {
-              return formatter;
-            }
-          };
-        }
+        columns: [{ title: 'Count' }],
+        rows: [[ { toString: () => formatter(4301021) } ]]
       }]
     });
 
@@ -44,14 +37,7 @@ describe('metric vis', function () {
           {title: '1st percentile of bytes'},
           {title: '99th percentile of bytes'}
         ],
-        rows: [[182, 445842.4634666484]],
-        aggConfig: function () {
-          return {
-            fieldFormatter: function () {
-              return formatter;
-            }
-          };
-        }
+        rows: [[ { toString: () => formatter(182) }, { toString: () => formatter(445842.4634666484) } ]]
       }]
     });
 

diff --git a/src/core_plugins/metric_vis/public/metric_vis_controller.js b/src/core_plugins/metric_vis/public/metric_vis_controller.js
@@ -17,23 +17,24 @@ module.controller('KbnMetricVisController', function ($scope, $element, Private)
   $scope.processTableGroups = function (tableGroups) {
     tableGroups.tables.forEach(function (table) {
       table.columns.forEach(function (column, i) {
-        const fieldFormatter = table.aggConfig(column).fieldFormatter();
-        let value = table.rows[0][i];
-
-        value = isInvalid(value) ? '?' : fieldFormatter(value);
+        const value = table.rows[0][i];
 
         metrics.push({
           label: column.title,
-          value: value
+          value: value.toString('html')
         });
       });
     });
   };
 
   $scope.$watch('esResponse', function (resp) {
     if (resp) {
+      const options = {
+        asAggConfigResults: true
+      };
+
       metrics.length = 0;
-      $scope.processTableGroups(tabifyAggResponse($scope.vis, resp));
+      $scope.processTableGroups(tabifyAggResponse($scope.vis, resp, options));
       $element.trigger('renderComplete');
     }
   });

diff --git a/src/fixtures/logstash_fields.js b/src/fixtures/logstash_fields.js
@@ -5,7 +5,7 @@ function stubbedLogstashFields() {
     //                                  |      |      |aggregatable
     //                                  |      |      |      |searchable
     // name                type         |      |      |      |     |metadata
-    ['bytes',             'number',     true,  true,  true,  true,  { count: 10 } ],
+    ['bytes',             'number',     true,  true,  true,  true,  { count: 10, docValues: true } ],
     ['ssl',               'boolean',    true,  true,  true,  true,  { count: 20 } ],
     ['@timestamp',        'date',       true,  true,  true,  true,  { count: 30 } ],
     ['time',              'date',       true,  true,  true,  true,  { count: 30 } ],
@@ -20,6 +20,7 @@ function stubbedLogstashFields() {
     ['geo.coordinates',   'geo_point',  true,  true,  true,  true ],
     ['extension',         'string',     true,  true,  true,  true ],
     ['machine.os',        'string',     true,  true,  true,  true ],
+    ['machine.os.raw',    'string',     true,  false, true,  true,  { docValues: true } ],
     ['geo.src',           'string',     true,  true,  true,  true ],
     ['_id',               'string',     false, false, true,  true ],
     ['_type',             'string',     false, false, true,  true ],
@@ -41,6 +42,7 @@ function stubbedLogstashFields() {
     ] = row;
 
     const {
+      docValues = false,
       count = 0,
       script,
       lang = script ? 'expression' : undefined,
@@ -50,6 +52,7 @@ function stubbedLogstashFields() {
     return {
       name,
       type,
+      doc_values: docValues,
       indexed,
       analyzed,
       aggregatable,

diff --git a/src/test_utils/stub_index_pattern.js b/src/test_utils/stub_index_pattern.js
@@ -8,6 +8,7 @@ import getComputedFields from 'ui/index_patterns/_get_computed_fields';
 import RegistryFieldFormatsProvider from 'ui/registry/field_formats';
 import IndexPatternsFlattenHitProvider from 'ui/index_patterns/_flatten_hit';
 import IndexPatternsFieldProvider from 'ui/index_patterns/_field';
+
 export default function (Private) {
   let fieldFormats = Private(RegistryFieldFormatsProvider);
   let flattenHit = Private(IndexPatternsFlattenHitProvider);