Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Data Grid Histograms #68359

Merged
merged 46 commits into from
Jun 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
202035a
[ML] Initial experimental version.
walterra Jun 5, 2020
b2f1f31
[ML] Adds button to toggle chart visibility.
walterra Jun 5, 2020
f1eb766
[ML] Fix hard coded colors.
walterra Jun 5, 2020
294eac6
[ML] Move data fetching to hook. Fix toggling column visibility. Move…
walterra Jun 8, 2020
4e2ef59
[ML] Tweak SCSS. Fetch data in outer hook. TS improvements.
walterra Jun 8, 2020
f3873bd
[ML] WIP: Fix column toggling when charts are enabled.
walterra Jun 9, 2020
9e4423a
[ML] Fix column toggling.
walterra Jun 9, 2020
b571b9c
[ML] Translate chart legend.
walterra Jun 9, 2020
34bfb7a
[ML] Fix column resizing.
walterra Jun 9, 2020
072b698
[ML] Only fetch charts data when they are visible.
walterra Jun 9, 2020
6d3a878
[ML] Return empty dataset for stats returning null.
walterra Jun 10, 2020
5f7df12
[ML] Tweak legend style.
walterra Jun 10, 2020
50de160
[ML] Add charts to more data grids. Fix non-aggregatable columns.
walterra Jun 10, 2020
6151122
[ML] Types cleanup.
walterra Jun 10, 2020
53b34be
[ML] Custom legend for boolean histograms.
walterra Jun 10, 2020
dfb6d11
[ML] Optimized data fetching for charts.
walterra Jun 10, 2020
4026c82
Merge branch 'master' into ml-data-grid-mini-charts
walterra Jun 15, 2020
c4acf11
[ML] Use EuiDataGrid's own display prop to render chart instead of ref.
walterra Jun 15, 2020
9364733
[ML] SCSS cleanup.
walterra Jun 15, 2020
ebcef61
[ML] Fix header alignment.
walterra Jun 15, 2020
1b37d3f
[ML] Tweak legend SCSS.
walterra Jun 15, 2020
3393f59
[ML] Fix fieldType check.
walterra Jun 16, 2020
e79ca57
[ML] Reduce INIT_MAX_COLUMNS from 20 to 10.
walterra Jun 16, 2020
e270a90
Merge branch 'master' into ml-data-grid-mini-charts
walterra Jun 16, 2020
17b8752
[ML] Use classNames instead of custom string.
walterra Jun 16, 2020
4a66f7d
[ML] i18n cleanup.
walterra Jun 16, 2020
c8a6980
[ML] Fix types. Set MAX_COLUMNS to 10. Fix to consider query update.
walterra Jun 16, 2020
730c8d3
[ML] Fix to consider query to outlier results. Fix if interval is 0.
walterra Jun 16, 2020
510f7b6
[ML] Adds chartsButtonVisible prop.
walterra Jun 16, 2020
813065d
[ML] Text tweak.
walterra Jun 16, 2020
4dadf58
Merge branch 'master' into ml-data-grid-mini-charts
walterra Jun 17, 2020
90b5939
[ML] Fix functional tests. Fix hook render loop.
walterra Jun 17, 2020
aaf6bdf
[ML] Fix data fetching when there are no numeric columns.
walterra Jun 17, 2020
ff8903d
[ML] Improved types. Display message for unsupported column types.
walterra Jun 17, 2020
8364f20
[ML] Fix boolean chart legend.
walterra Jun 17, 2020
fbc9221
[ML] Improve stringHash test.
walterra Jun 17, 2020
a717964
[ML] Fix state/loop issue.
walterra Jun 17, 2020
6c6d9f7
Merge branch 'master' into ml-data-grid-mini-charts
walterra Jun 17, 2020
8570a4b
[ML] Improve legend text when chart not available or no data available.
walterra Jun 17, 2020
eee4bfa
[ML] Remove unneeded import and margin.
walterra Jun 17, 2020
0deff76
[ML] Add e2e tests.
walterra Jun 18, 2020
1bc0bca
[ML] e2e tweaks.
walterra Jun 18, 2020
f9a1399
[ML] Fix error messages.
walterra Jun 18, 2020
4da1352
[ML] Fix types for api/ml.esSearch.
walterra Jun 18, 2020
82447ae
[ML] Remove unneeded try/catch blocks.
walterra Jun 18, 2020
2f6f642
[ML] Return an empty legend string instead of throwing an error.
walterra Jun 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 2 additions & 14 deletions x-pack/plugins/ml/common/util/group_color_utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

import euiVars from '@elastic/eui/dist/eui_theme_dark.json';

import { stringHash } from './string_utils';

const COLORS = [
euiVars.euiColorVis0,
euiVars.euiColorVis1,
Expand Down Expand Up @@ -33,17 +35,3 @@ export function tabColor(name: string): string {
return colorMap[name];
}
}

function stringHash(str: string): number {
let hash = 0;
let chr = 0;
if (str.length === 0) {
return hash;
}
for (let i = 0; i < str.length; i++) {
chr = str.charCodeAt(i);
hash = (hash << 5) - hash + chr; // eslint-disable-line no-bitwise
hash |= 0; // eslint-disable-line no-bitwise
}
return hash < 0 ? hash * -2 : hash;
}
10 changes: 9 additions & 1 deletion x-pack/plugins/ml/common/util/string_utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { renderTemplate, getMedianStringLength } from './string_utils';
import { renderTemplate, getMedianStringLength, stringHash } from './string_utils';

const strings: string[] = [
'foo',
Expand Down Expand Up @@ -46,4 +46,12 @@ describe('ML - string utils', () => {
expect(result).toBe(0);
});
});

describe('stringHash', () => {
test('should return a unique number based off a string', () => {
const hash1 = stringHash('the-string-1');
const hash2 = stringHash('the-string-2');
expect(hash1).not.toBe(hash2);
});
});
});
17 changes: 17 additions & 0 deletions x-pack/plugins/ml/common/util/string_utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,20 @@ export function getMedianStringLength(strings: string[]) {
const sortedStringLengths = strings.map((s) => s.length).sort((a, b) => a - b);
return sortedStringLengths[Math.floor(sortedStringLengths.length / 2)] || 0;
}

/**
* Creates a deterministic number based hash out of a string.
*/
export function stringHash(str: string): number {
let hash = 0;
let chr = 0;
if (str.length === 0) {
return hash;
}
for (let i = 0; i < str.length; i++) {
chr = str.charCodeAt(i);
hash = (hash << 5) - hash + chr; // eslint-disable-line no-bitwise
hash |= 0; // eslint-disable-line no-bitwise
}
return hash < 0 ? hash * -2 : hash;
}
darnautov marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
.mlDataGridChart__histogram {
width: 100%;
height: $euiSizeXL + $euiSizeXXL;
}

.mlDataGridChart__legend {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one can probably be improved a little.

@include euiTextTruncate;
@include euiFontSizeXS

Then you won't need all the overflow stuff, and can get rid of the line-heights and sizing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the hint, fixed in 1b37d3f.

@include euiTextTruncate;
@include euiFontSizeXS;

color: $euiColorMediumShade;
display: block;
overflow-x: hidden;
margin: $euiSizeXS 0px 0px 0px;
font-style: italic;
font-weight: normal;
text-align: left;
}

.mlDataGridChart__legend--numeric {
text-align: right;
}

.mlDataGridChart__legendBoolean {
width: 100%;
td { text-align: center }
}

/* Override to align column header to bottom of cell when no chart is available */
.mlDataGrid .euiDataGridHeaderCell__content {
margin-top: auto;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import React, { FC } from 'react';
import classNames from 'classnames';

import { BarSeries, Chart, Settings } from '@elastic/charts';
import { EuiDataGridColumn } from '@elastic/eui';

import './column_chart.scss';

import { isUnsupportedChartData, useColumnChart, ChartData } from './use_column_chart';

interface Props {
chartData: ChartData;
columnType: EuiDataGridColumn;
dataTestSubj: string;
}

export const ColumnChart: FC<Props> = ({ chartData, columnType, dataTestSubj }) => {
const { data, legendText, xScaleType } = useColumnChart(chartData, columnType);

return (
<div data-test-subj={dataTestSubj}>
{!isUnsupportedChartData(chartData) && data.length > 0 && (
peteharverson marked this conversation as resolved.
Show resolved Hide resolved
<div className="mlDataGridChart__histogram" data-test-subj={`${dataTestSubj}-histogram`}>
<Chart>
<Settings
theme={{
background: { color: 'transparent' },
chartMargins: {
left: 0,
right: 0,
top: 0,
bottom: 1,
},
chartPaddings: {
left: 0,
right: 0,
top: 0,
bottom: 0,
},
scales: { barsPadding: 0.1 },
}}
/>
<BarSeries
id="histogram"
name="count"
xScaleType={xScaleType}
yScaleType="linear"
xAccessor="key"
yAccessors={['doc_count']}
styleAccessor={(d) => d.datum.color}
data={data}
/>
</Chart>
</div>
)}
<div
className={classNames('mlDataGridChart__legend', {
'mlDataGridChart__legend--numeric': columnType.schema === 'number',
})}
data-test-subj={`${dataTestSubj}-legend`}
>
{legendText}
</div>
<div data-test-subj={`${dataTestSubj}-id`}>{columnType.id}</div>
</div>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,19 @@ import {
EuiDataGridStyle,
} from '@elastic/eui';

import { i18n } from '@kbn/i18n';

import { CoreSetup } from 'src/core/public';

import {
IndexPattern,
IFieldType,
ES_FIELD_TYPES,
KBN_FIELD_TYPES,
} from '../../../../../../../src/plugins/data/public';

import { extractErrorMessage } from '../../../../common/util/errors';

import {
BASIC_NUMERICAL_TYPES,
EXTENDED_NUMERICAL_TYPES,
Expand All @@ -37,7 +43,7 @@ import { mlFieldFormatService } from '../../services/field_format_service';

import { DataGridItem, IndexPagination, RenderCellValue } from './types';

export const INIT_MAX_COLUMNS = 20;
export const INIT_MAX_COLUMNS = 10;

export const euiDataGridStyle: EuiDataGridStyle = {
border: 'all',
Expand Down Expand Up @@ -102,6 +108,8 @@ export const getDataGridSchemasFromFieldTypes = (fieldTypes: FieldTypes, results
case 'boolean':
schema = 'boolean';
break;
case 'text':
schema = NON_AGGREGATABLE;
}

if (
Expand All @@ -122,7 +130,10 @@ export const getDataGridSchemasFromFieldTypes = (fieldTypes: FieldTypes, results
});
};

export const getDataGridSchemaFromKibanaFieldType = (field: IFieldType | undefined) => {
export const NON_AGGREGATABLE = 'non-aggregatable';
export const getDataGridSchemaFromKibanaFieldType = (
field: IFieldType | undefined
): string | undefined => {
// Built-in values are ['boolean', 'currency', 'datetime', 'numeric', 'json']
// To fall back to the default string schema it needs to be undefined.
let schema;
Expand All @@ -143,6 +154,10 @@ export const getDataGridSchemaFromKibanaFieldType = (field: IFieldType | undefin
break;
}

if (schema === undefined && field?.aggregatable === false) {
return NON_AGGREGATABLE;
}

return schema;
};

Expand Down Expand Up @@ -289,3 +304,17 @@ export const multiColumnSortFactory = (sortingColumns: EuiDataGridSorting['colum

return sortFn;
};

export const showDataGridColumnChartErrorMessageToast = (
e: any,
toastNotifications: CoreSetup['notifications']['toasts']
) => {
const error = extractErrorMessage(e);

toastNotifications.addDanger(
i18n.translate('xpack.ml.dataGrid.columnChart.ErrorMessageToast', {
defaultMessage: 'An error occurred fetching the histogram charts data: {error}',
values: { error: error !== '' ? error : e },
})
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import React, { memo, useEffect, FC } from 'react';
import { i18n } from '@kbn/i18n';

import {
EuiButtonEmpty,
EuiButtonIcon,
EuiCallOut,
EuiCodeBlock,
Expand All @@ -27,6 +28,8 @@ import { INDEX_STATUS } from '../../data_frame_analytics/common';

import { euiDataGridStyle, euiDataGridToolbarSettings } from './common';
import { UseIndexDataReturnType } from './types';
// TODO Fix row hovering + bar highlighting
// import { hoveredRow$ } from './column_chart';
Comment on lines +31 to +32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commented code 👀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be solved in a follow up. Originally this code was used to work with EuiBasicTable which had a callback when hovering table rows. EuiDataGrid doesn't come with that out of the box so it needs more investigation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure it's fine to resolve in a follow-up, but with VCS there is no need in keeping the commented code 🙂


export const DataGridTitle: FC<{ title: string }> = ({ title }) => (
<EuiTitle size="xs">
Expand Down Expand Up @@ -54,7 +57,9 @@ type Props = PropsWithHeader | PropsWithoutHeader;
export const DataGrid: FC<Props> = memo(
(props) => {
const {
columns,
chartsVisible,
chartsButtonVisible,
columnsWithCharts,
dataTestSubj,
errorMessage,
invalidSortingColumnns,
Expand All @@ -70,9 +75,18 @@ export const DataGrid: FC<Props> = memo(
status,
tableItems: data,
toastNotifications,
toggleChartVisibility,
visibleColumns,
} = props;

// TODO Fix row hovering + bar highlighting
// const getRowProps = (item: any) => {
// return {
// onMouseOver: () => hoveredRow$.next(item),
// onMouseLeave: () => hoveredRow$.next(null),
// };
// };

Comment on lines +82 to +89
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commented code

useEffect(() => {
if (invalidSortingColumnns.length > 0) {
invalidSortingColumnns.forEach((columnId) => {
Expand Down Expand Up @@ -162,22 +176,50 @@ export const DataGrid: FC<Props> = memo(
<EuiSpacer size="m" />
</div>
)}
<EuiDataGrid
aria-label={isWithHeader(props) ? props.title : ''}
columns={columns}
columnVisibility={{ visibleColumns, setVisibleColumns }}
gridStyle={euiDataGridStyle}
rowCount={rowCount}
renderCellValue={renderCellValue}
sorting={{ columns: sortingColumns, onSort }}
toolbarVisibility={euiDataGridToolbarSettings}
pagination={{
...pagination,
pageSizeOptions: [5, 10, 25],
onChangeItemsPerPage,
onChangePage,
}}
/>
<div className="mlDataGrid">
<EuiDataGrid
aria-label={isWithHeader(props) ? props.title : ''}
columns={columnsWithCharts.map((c) => {
c.initialWidth = 165;
return c;
})}
columnVisibility={{ visibleColumns, setVisibleColumns }}
gridStyle={euiDataGridStyle}
rowCount={rowCount}
renderCellValue={renderCellValue}
sorting={{ columns: sortingColumns, onSort }}
toolbarVisibility={{
...euiDataGridToolbarSettings,
...(chartsButtonVisible
? {
additionalControls: (
<EuiButtonEmpty
aria-checked={chartsVisible}
className={`euiDataGrid__controlBtn${
chartsVisible ? ' euiDataGrid__controlBtn--active' : ''
}`}
data-test-subj={`${dataTestSubj}HistogramButton`}
size="xs"
iconType="visBarVertical"
color="text"
onClick={toggleChartVisibility}
>
{i18n.translate('xpack.ml.dataGrid.histogramButtonText', {
defaultMessage: 'Histogram charts',
})}
</EuiButtonEmpty>
),
}
: {}),
}}
pagination={{
...pagination,
pageSizeOptions: [5, 10, 25],
onChangeItemsPerPage,
onChangePage,
}}
/>
</div>
</div>
);
},
Expand All @@ -186,7 +228,7 @@ export const DataGrid: FC<Props> = memo(

function pickProps(props: Props) {
return [
props.columns,
props.columnsWithCharts,
props.dataTestSubj,
props.errorMessage,
props.invalidSortingColumnns,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ export {
getDataGridSchemaFromKibanaFieldType,
getFieldsFromKibanaIndexPattern,
multiColumnSortFactory,
showDataGridColumnChartErrorMessageToast,
useRenderCellValue,
} from './common';
export { fetchChartsData, ChartData } from './use_column_chart';
export { useDataGrid } from './use_data_grid';
export { DataGrid } from './data_grid';
export {
Expand Down
Loading