[Lens][TSVB] Ad-hoc dataViews for index pattern string mode in TSVB.

[Kuznietsov]

Summary

Completes part of #138236.
Added support of ad-hoc dataViews while converting TSVB visualizations, when index pattern string mode is turned on.

[elasticmachine]

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

[flash1293]

Doesn't work right for "overwrite data view":

Becomes:

[dimaanj]

Tested locally, changed logic when navigate from Discover works correctly.

[dimaanj]

Checked on main branch. Typing data view name in string mode doesn't give an error toast on each change.

tsvb.mp4

[VladLasitsa]

Found a problem: as in each change of model, we execute convert to lens logic, we will create a new ad hoc dataview on each typing action in index pattern field. It will lead to creation a lot of useless ad hoc dataviews in cache. Two possible way to resolve problem:

• Move logic related to create ad hoc dataview to lens side
• Create ad hoc dataview only when user click on "Edit in Lens" button

[flash1293]

In general it works fine, but there are too many data view lookups going on - every change to the configuration causes a bunch of them if index pattern string mode is used (which is a common thing).

This is what happens if multiple visualizations with index pattern string mode are used on a dashboard and the user hits refresh (after the chart loaded successfully previously):

For regular data views, there are no calls like this at all. Somehow we need to make sure these are cached.

[flash1293]

See above

[Kuznietsov]

In general it works fine, but there are too many data view lookups going on - every change to the configuration causes a bunch of them if index pattern string mode is used (which is a common thing)

@flash1293, that is how convertors are built and observation of "canNavigateToLens" on every change, and there is no way of fixing this. Can you explain, please, what you expect to see as a solution?

[flash1293]

@Kunzetsov You already introduced an "adhoc data view cache" in the conversion logic, right? Could we just keep it around instead of resetting it?

[Kuznietsov]

@Kunzetsov You already introduced an "adhoc data view cache" in the conversion logic, right? Could we just keep it around instead of resetting it?

@flash1293, To be honest, we can't.

1. We should save ad-hoc dataviews only after click on "Edit in lens".
2. If the user is moving to the other part of the Kibana, all the created ad-hocs have to be cleaned up from the global ad-hoc dataviews cache. And after converting (checking if it is possible), all the dataViews, created by a call, are cleaned up.
3. If user is converting the vis to Lens, all the ad-hoc dataViews should be moved to Lens and kept in a global state.
4. Every change is calling "convertToLens", to detect if converting is possible and should be cleaned up after execution, because nobody guarantee, that user will convert the vis.
5. Even if we'll keep the cache between calls, we have to make an API call for checking if such index with the full name exists to fetch the time field (so, calls to the API will not reduce too much).

[flash1293]

Fair considerations @Kunzetsov As it turned out the same issue occurs in other places as well, I will create an issue to solve it in the right place. However, we can't merge this before it's fixed as it will wreak havoc on more complex TSVB-heavy dashboards (performance wise)

[Kuznietsov]

Fair considerations @Kunzetsov As it turned out the same issue occurs in other places as well, I will create an issue to solve it in the right place. However, we can't merge this before it's fixed as it will wreak havoc on more complex TSVB-heavy dashboards (performance wise)

@flash1293, ok, will be waiting for the updates.

[VladLasitsa]

@elasticmachine merge upstream

[kibanamachine]

merge conflict between base and head

[VladLasitsa]

@elasticmachine merge upstream

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: b1c3aaf

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
visTypeTimeseries	383	381	-2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lens	1.3MB	1.3MB	+244.0B
visTypeTimeseries	509.2KB	509.7KB	+486.0B
total			+730.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
visTypeTimeseries	19.7KB	19.7KB	-36.0B

Unknown metric groups

ESLint disabled in files

id	before	after	diff
osquery	1	2	+1

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	19	21	+2
fleet	59	65	+6
osquery	108	113	+5
securitySolution	440	446	+6
total			+19

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	20	22	+2
fleet	67	73	+6
osquery	109	115	+6
securitySolution	517	523	+6
total			+20

History

• 💔 Build #86596 failed fe37115
• 💚 Build #86313 succeeded 2e4a088
• 💚 Build #85814 succeeded 5159470
• 💚 Build #83839 succeeded 3ad2a09
• 💔 Build #83808 failed 2888be6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @VladLasitsa @Kunzetsov

[flash1293]

Tested around and everything works fine for me now, LGTM