Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement DataViewLazy / lazy field loading #178594

Closed
mattkime opened this issue Mar 13, 2024 · 2 comments
Closed

Implement DataViewLazy / lazy field loading #178594

mattkime opened this issue Mar 13, 2024 · 2 comments
Assignees
@mattkime
Labels
Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:large Large Level of Effort Meta Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@mattkime
Copy link
Contributor

mattkime commented Mar 13, 2024 •
edited
Loading

More detailed breakdown of whats in #167595

  1. Create AbstractDataView class. Contains all DataView code that doesn't rely on the field list. - [DataViews] Create AbstractDataView class #167736
  2. Implement DataViewLazy, use by DataViews REST APIs - [data views] DataViewLazy implementation phase 1 #173948
  3. Add SearchSource compatibility
  4. Use DataViewLazy in DataView Management
  5. Use DataViewLazy in Discover
  6. Roll out to solutions and kibana apps

other notable changes using DataViewLazy over DataView

  • won't throw if index pattern fails to match
  • by default toSpec doesn't return fields, must pass in parameters for requested fields

Skipped for now

  • isTSDBMode gets all fields. Should find more efficient method
  • getting scripted fields and runtime fields don't support wildcard. getting mapped fields does
  • individual field changes are shared across getFields calls, but field additions and removals aren't shared across fetches
  • not examining eTag to skip parsing field list
@mattkime mattkime added Feature:Data Views Data Views code and UI - index patterns before 8.0 Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. labels Mar 13, 2024
@mattkime mattkime self-assigned this Mar 13, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@mattkime mattkime mentioned this issue Mar 13, 2024
Merged
@mattkime mattkime added Meta loe:large Large Level of Effort impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. labels Mar 13, 2024
mattkime added a commit that referenced this issue Apr 3, 2024
@mattkime @kibanamachine
[data views] DataViewLazy implementation phase 1 (#173948)
e7a3e2c 
## Summary

Part 1 of #178594

Initial implementation of DataViewLazy class - an alternative DataView
implementation that doesn't load the full field list on initial
creation.

```
Implements DataViewLazy (variations from DataView class)
- getFields ({ mapped = true, scripted = true, runtime = true, fieldName, forceRefresh = false, unmapped, indexFilter, metaFields = true }) => 
    Promise<{ getFieldMap: () => Record<string, DataViewField>; getFieldMapSorted: () => Record<string, DataViewField> }>
- getRuntimeFields({ fieldName = ['*'] }) => Record<string, DataViewField>
- getScriptedFields({ fieldName = ['*'] }) => Record<string, DataViewField>
- getFieldByName (string) => Promise<DataViewField>
- toSpec (same params as getFields) => Promise<DataViewSpec>

other notable changes using DataViewLazy over DataView
- won't throw if index pattern fails to match
- by default toSpec doesn't return fields, must pass in parameters for requested fields

---

DataViews service additions
- toDataView: (toDataView: DataViewLazy) => Promise<DataView>;
- toDataViewLazy: (dataView: DataView) => Promise<DataViewLazy>;
- getAllDataViewLazy: () => Promise<DataViewLazy[]>;
- getDataViewLazy: (id: string) => Promise<DataViewLazy>;
- createDataViewLazy: (spec: DataViewSpec) => Promise<DataViewLazy>;
- createAndSaveDataViewLazy: (spec: DataViewSpec, override?: boolean) => Promise<DataViewLazy>;
- getDefaultDataViewLazy: () => Promise<DataViewLazy | null>;
```

Closes #176426 and
#167750

---------

Co-authored-by: kibanamachine <[email protected]>
@mattkime
Copy link
Contributor Author

Closing in favor of #167595

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattkime mattkime

Labels
Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:large Large Level of Effort Meta Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mattkime @elasticmachine