Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataGrid] Filtering performance: use Set() when applicable #10068

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ const getAggregationCellValue = ({

const values: any[] = [];
rowIds.forEach((rowId) => {
if (aggregationRowsScope === 'filtered' && filteredRowsLookup[rowId] === false) {
if (aggregationRowsScope === 'filtered' && filteredRowsLookup.has(rowId)) {
return;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ export const filterRowTreeFromGroupingColumns = (
params: FilterRowTreeFromTreeDataParams,
): Omit<GridFilterState, 'filterModel'> => {
const { apiRef, rowTree, isRowMatchingFilters, filterModel } = params;
const filteredRowsLookup: Record<GridRowId, boolean> = {};
const filteredRowsLookup = new Set<GridRowId>();
const filteredDescendantCountLookup: Record<GridRowId, number> = {};
const filterCache = {};

Expand Down Expand Up @@ -141,7 +141,9 @@ export const filterRowTreeFromGroupingColumns = (
}
}

filteredRowsLookup[node.id] = isPassingFiltering;
if (!isPassingFiltering) {
filteredRowsLookup.add(node.id);
}

if (!isPassingFiltering) {
return 0;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import {
createRowTree,
updateRowTree,
RowTreeBuilderGroupingCriterion,
getVisibleRowsLookup,
getHiddenRowsLookup,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, making that change lost nearly all of the performance improvements observed in the first step.

I don't understand what caused the loss of the performance improvements.
First, you tried using Set to keep all the visible rows.
Then, you reversed it to only keep the hidden rows. Did this increase the number of records in the Set drastically?
Or is it something else that impacted performance?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also didn't make sense to me, but I haven't had enough time to investigate further.

} from '@mui/x-data-grid-pro/internals';
import { DataGridPremiumProcessedProps } from '../../../models/dataGridPremiumProps';
import {
Expand Down Expand Up @@ -237,8 +237,8 @@ export const useGridRowGroupingPreProcessors = (
useGridRegisterStrategyProcessor(
apiRef,
ROW_GROUPING_STRATEGY,
'visibleRowsLookupCreation',
getVisibleRowsLookup,
'hiddenRowsLookupCreation',
getHiddenRowsLookup,
);

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ export const filterRowTreeFromTreeData = (
params: FilterRowTreeFromTreeDataParams,
): Omit<GridFilterState, 'filterModel'> => {
const { apiRef, rowTree, disableChildrenFiltering, isRowMatchingFilters } = params;
const filteredRowsLookup: Record<GridRowId, boolean> = {};
const filteredRowsLookup = new Set<GridRowId>();
const filteredDescendantCountLookup: Record<GridRowId, number> = {};
const filterCache = {};

Expand Down Expand Up @@ -94,7 +94,9 @@ export const filterRowTreeFromTreeData = (
}
}

filteredRowsLookup[node.id] = shouldPassFilters;
if (!shouldPassFilters) {
filteredRowsLookup.add(node.id);
}

if (!shouldPassFilters) {
return 0;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import {
} from '../../../utils/tree/models';
import { sortRowTree } from '../../../utils/tree/sortRowTree';
import { updateRowTree } from '../../../utils/tree/updateRowTree';
import { getVisibleRowsLookup } from '../../../utils/tree/utils';
import { getHiddenRowsLookup } from '../../../utils/tree/utils';

export const useGridTreeDataPreProcessors = (
privateApiRef: React.MutableRefObject<GridPrivateApiPro>,
Expand Down Expand Up @@ -216,8 +216,8 @@ export const useGridTreeDataPreProcessors = (
useGridRegisterStrategyProcessor(
privateApiRef,
TREE_DATA_STRATEGY,
'visibleRowsLookupCreation',
getVisibleRowsLookup,
'hiddenRowsLookupCreation',
getHiddenRowsLookup,
);

/**
Expand Down
2 changes: 1 addition & 1 deletion packages/grid/x-data-grid-pro/src/internals/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,5 @@ export type {
export { createRowTree } from '../utils/tree/createRowTree';
export { updateRowTree } from '../utils/tree/updateRowTree';
export { sortRowTree } from '../utils/tree/sortRowTree';
export { insertNodeInTree, removeNodeFromTree, getVisibleRowsLookup } from '../utils/tree/utils';
export { insertNodeInTree, removeNodeFromTree, getHiddenRowsLookup } from '../utils/tree/utils';
export type { RowTreeBuilderGroupingCriterion } from '../utils/tree/models';
19 changes: 11 additions & 8 deletions packages/grid/x-data-grid-pro/src/utils/tree/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -217,21 +217,21 @@ export const createUpdatedGroupsManager = (): GridRowTreeUpdatedGroupsManager =>
},
});

export const getVisibleRowsLookup = ({
export const getHiddenRowsLookup = ({
tree,
filteredRowsLookup,
}: {
tree: GridRowsState['tree'];
filteredRowsLookup: GridFilterState['filteredRowsLookup'];
}) => {
if (!filteredRowsLookup) {
return {};
return new Set<GridRowId>();
}

const visibleRowsLookup: Record<GridRowId, boolean> = {};
const hiddenRowsLookup = new Set<GridRowId>();

const handleTreeNode = (node: GridTreeNode, areAncestorsExpanded: boolean) => {
const isPassingFiltering = filteredRowsLookup[node.id];
const isPassingFiltering = !filteredRowsLookup.has(node.id);

if (node.type === 'group') {
node.children.forEach((childId) => {
Expand All @@ -240,12 +240,15 @@ export const getVisibleRowsLookup = ({
});
}

visibleRowsLookup[node.id] = isPassingFiltering && areAncestorsExpanded;
if (!(isPassingFiltering && areAncestorsExpanded)) {
hiddenRowsLookup.add(node.id);
}

// TODO rows v6: Should we keep storing the visibility status of footer independently or rely on the group visibility in the selector ?
if (node.type === 'group' && node.footerId != null) {
visibleRowsLookup[node.footerId] =
isPassingFiltering && areAncestorsExpanded && !!node.childrenExpanded;
if (!(isPassingFiltering && areAncestorsExpanded && !!node.childrenExpanded)) {
hiddenRowsLookup.add(node.footerId);
}
}
};

Expand All @@ -256,5 +259,5 @@ export const getVisibleRowsLookup = ({
handleTreeNode(node, true);
}
}
return visibleRowsLookup;
return hiddenRowsLookup;
};
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import {
GridFilteringMethodParams,
GridFilteringMethodValue,
GridFilterState,
GridVisibleRowsLookupState,
GridHiddenRowsLookupState,
} from '../../features/filter/gridFilterState';
import {
GridSortingMethodParams,
Expand Down Expand Up @@ -35,13 +35,13 @@ export interface GridStrategyProcessingLookup {
params: GridSortingMethodParams;
value: GridSortingMethodValue;
};
visibleRowsLookupCreation: {
hiddenRowsLookupCreation: {
group: 'rowTree';
params: {
tree: GridRowsState['tree'];
filteredRowsLookup: GridFilterState['filteredRowsLookup'];
};
value: GridVisibleRowsLookupState;
value: GridHiddenRowsLookupState;
};
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export const GRID_STRATEGIES_PROCESSORS: {
rowTreeCreation: 'rowTree',
filtering: 'rowTree',
sorting: 'rowTree',
visibleRowsLookupCreation: 'rowTree',
hiddenRowsLookupCreation: 'rowTree',
};

type UntypedStrategyProcessors = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export const gridQuickFilterValuesSelector = createSelector(
* @category Visible rows
* @ignore - do not document.
*/
export const gridVisibleRowsLookupSelector = (state: GridStateCommunity) => state.visibleRowsLookup;
export const gridHiddenRowsLookupSelector = (state: GridStateCommunity) => state.hiddenRowsLookup;

/**
* @category Filtering
Expand All @@ -58,10 +58,9 @@ export const gridFilteredDescendantCountLookupSelector = createSelector(
* @category Filtering
*/
export const gridExpandedSortedRowEntriesSelector = createSelectorMemoized(
gridVisibleRowsLookupSelector,
gridHiddenRowsLookupSelector,
gridSortedRowEntriesSelector,
(visibleRowsLookup, sortedRows) =>
sortedRows.filter((row) => visibleRowsLookup[row.id] !== false),
(hiddenRowsLookup, sortedRows) => sortedRows.filter((row) => !hiddenRowsLookup.has(row.id)),
);

/**
Expand All @@ -82,8 +81,7 @@ export const gridExpandedSortedRowIdsSelector = createSelectorMemoized(
export const gridFilteredSortedRowEntriesSelector = createSelectorMemoized(
gridFilteredRowsLookupSelector,
gridSortedRowEntriesSelector,
(filteredRowsLookup, sortedRows) =>
sortedRows.filter((row) => filteredRowsLookup[row.id] !== false),
(filteredRowsLookup, sortedRows) => sortedRows.filter((row) => !filteredRowsLookup.has(row.id)),
);

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,10 @@ export interface GridFilterState {

/**
* Filtering status for each row.
* A row is filtered if it is passing the filters, whether its parents are expanded or not.
* If a row is not registered in this lookup, it is filtered.
* This is the equivalent of the `visibleRowsLookup` if all the groups were expanded.
* This contains the rows that are filtered *out*, which means they did not pass the filters.
* This is the equivalent of the `hiddenRowsLookup` if all the groups were expanded.
*/
filteredRowsLookup: Record<GridRowId, boolean>;
filteredRowsLookup: Set<GridRowId>;
/**
* Amount of descendants that are passing the filters.
* For the Tree Data, it includes all the intermediate depth levels (= amount of children + amount of grand children + ...).
Expand Down Expand Up @@ -62,4 +61,4 @@ export type GridFilteringMethodValue = Omit<GridFilterState, 'filterModel'>;
* A row is visible if it is passing the filters AND if its parents are expanded.
* If a row is not registered in this lookup, it is visible.
*/
export type GridVisibleRowsLookupState = Record<GridRowId, boolean>;
export type GridHiddenRowsLookupState = Set<GridRowId>;
Original file line number Diff line number Diff line change
Expand Up @@ -44,23 +44,23 @@ export const filterStateInitializer: GridStateInitializer<
...state,
filter: {
filterModel: sanitizeFilterModel(filterModel, props.disableMultipleColumnsFiltering, apiRef),
filteredRowsLookup: {},
filteredRowsLookup: new Set(),
filteredDescendantCountLookup: {},
},
visibleRowsLookup: {},
hiddenRowsLookup: new Set(),
};
};

const getVisibleRowsLookup: GridStrategyProcessor<'visibleRowsLookupCreation'> = (params) => {
// For flat tree, the `visibleRowsLookup` and the `filteredRowsLookup` are equals since no row is collapsed.
const getHiddenRowsLookup: GridStrategyProcessor<'hiddenRowsLookupCreation'> = (params) => {
// For flat tree, the `hiddenRowsLookup` and the `filteredRowsLookup` are equals since no row is collapsed.
return params.filteredRowsLookup;
};

function getVisibleRowsLookupState(
function getHiddenRowsLookupState(
apiRef: React.MutableRefObject<GridPrivateApiCommunity>,
state: GridStateCommunity,
) {
return apiRef.current.applyStrategyProcessor('visibleRowsLookupCreation', {
return apiRef.current.applyStrategyProcessor('hiddenRowsLookupCreation', {
tree: state.rows.tree,
filteredRowsLookup: state.filter.filteredRowsLookup,
});
Expand Down Expand Up @@ -123,11 +123,11 @@ export const useGridFilter = (
},
};

const visibleRowsLookupState = getVisibleRowsLookupState(apiRef, newState);
const hiddenRowsLookupState = getHiddenRowsLookupState(apiRef, newState);

return {
...newState,
visibleRowsLookup: visibleRowsLookupState,
hiddenRowsLookup: hiddenRowsLookupState,
};
});
apiRef.current.publishEvent('filteredRowsSet');
Expand Down Expand Up @@ -403,13 +403,13 @@ export const useGridFilter = (
(params) => {
if (props.filterMode !== 'client' || !params.isRowMatchingFilters) {
return {
filteredRowsLookup: {},
filteredRowsLookup: new Set<GridRowId>(),
filteredDescendantCountLookup: {},
};
}

const dataRowIdToModelLookup = gridRowsLookupSelector(apiRef);
const filteredRowsLookup: Record<GridRowId, boolean> = {};
const filteredRowsLookup = new Set<GridRowId>();
const { isRowMatchingFilters } = params;
const filterCache = {};

Expand All @@ -433,13 +433,15 @@ export const useGridFilter = (
filterCache,
);

filteredRowsLookup[id] = isRowPassing;
if (!isRowPassing) {
filteredRowsLookup.add(id);
}
}

const footerId = 'auto-generated-group-footer-root';
const footer = dataRowIdToModelLookup[footerId];
if (footer) {
filteredRowsLookup[footerId] = true;
filteredRowsLookup.add(footerId);
}

return {
Expand All @@ -458,8 +460,8 @@ export const useGridFilter = (
useGridRegisterStrategyProcessor(
apiRef,
GRID_DEFAULT_STRATEGY,
'visibleRowsLookupCreation',
getVisibleRowsLookup,
'hiddenRowsLookupCreation',
getHiddenRowsLookup,
);

/**
Expand Down Expand Up @@ -488,11 +490,11 @@ export const useGridFilter = (
[apiRef],
);

const updateVisibleRowsLookupState = React.useCallback(() => {
const updateHiddenRowsLookupState = React.useCallback(() => {
apiRef.current.setState((state) => {
return {
...state,
visibleRowsLookup: getVisibleRowsLookupState(apiRef, state),
hiddenRowsLookup: getHiddenRowsLookupState(apiRef, state),
};
});
apiRef.current.forceUpdate();
Expand All @@ -503,7 +505,7 @@ export const useGridFilter = (
useGridApiEventHandler(apiRef, 'rowsSet', updateFilteredRows);
useGridApiEventHandler(apiRef, 'columnsChange', handleColumnsChange);
useGridApiEventHandler(apiRef, 'activeStrategyProcessorChange', handleStrategyProcessorChange);
useGridApiEventHandler(apiRef, 'rowExpansionChange', updateVisibleRowsLookupState);
useGridApiEventHandler(apiRef, 'rowExpansionChange', updateHiddenRowsLookupState);
useGridApiEventHandler(apiRef, 'columnVisibilityModelChange', () => {
const filterModel = gridFilterModelSelector(apiRef);
if (filterModel.quickFilterValues && filterModel.quickFilterExcludeHiddenColumns) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ export const useGridRows = (

if (applyFiltering) {
const filteredRowsLookup = gridFilteredRowsLookupSelector(apiRef);
children = children.filter((childId) => filteredRowsLookup[childId] !== false);
children = children.filter((childId) => !filteredRowsLookup.has(childId));
}

return children;
Expand Down
4 changes: 2 additions & 2 deletions packages/grid/x-data-grid/src/models/gridStateCommunity.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ import type { GridRowsMetaState } from '../hooks/features/rows/gridRowsMetaState
import type { GridEditingState } from './gridEditRowModel';
import { GridHeaderFilteringState } from './gridHeaderFilteringModel';
import type { GridRowSelectionModel } from './gridRowSelectionModel';
import type { GridVisibleRowsLookupState } from '../hooks/features/filter/gridFilterState';
import type { GridHiddenRowsLookupState } from '../hooks/features/filter/gridFilterState';

/**
* The state of `DataGrid`.
*/
export interface GridStateCommunity {
rows: GridRowsState;
visibleRowsLookup: GridVisibleRowsLookupState;
hiddenRowsLookup: GridHiddenRowsLookupState;
rowsMeta: GridRowsMetaState;
editRows: GridEditingState;
headerFiltering: GridHeaderFilteringState;
Expand Down
2 changes: 1 addition & 1 deletion scripts/x-data-grid-premium.exports.json
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@
{ "name": "GridHeaderFilterMenu", "kind": "Function" },
{ "name": "GridHeaderFilterMenuContainer", "kind": "Function" },
{ "name": "GridHeaderSelectionCheckboxParams", "kind": "Interface" },
{ "name": "gridHiddenRowsLookupSelector", "kind": "Variable" },
{ "name": "GridIconSlotsComponent", "kind": "Interface" },
{ "name": "GridInitialState", "kind": "TypeAlias" },
{ "name": "GridInputRowSelectionModel", "kind": "TypeAlias" },
Expand Down Expand Up @@ -574,7 +575,6 @@
{ "name": "GridVisibilityOffIcon", "kind": "Variable" },
{ "name": "gridVisibleColumnDefinitionsSelector", "kind": "Variable" },
{ "name": "gridVisibleColumnFieldsSelector", "kind": "Variable" },
{ "name": "gridVisibleRowsLookupSelector", "kind": "Variable" },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels risky TBH. While gridVisibleRowsLookupSelector is not documented, it's exported and not marked as unstable/experimental.
Should we wait with this PR until v7 alpha?

{ "name": "GridWorkspacesIcon", "kind": "Variable" },
{ "name": "heIL", "kind": "Variable" },
{ "name": "huHU", "kind": "Variable" },
Expand Down
Loading