[Lens] Add custom formatter within the Lens editor #158468

dej611 · 2023-05-25T15:30:10Z

Summary

Fixes #157900

Goal of this PR:

simplify Metric formatting logic (only duration formats are now "enhanced")
add a new Compact option for the existing Lens formatters (Bytes and Bits excluded as they compact already)

provide a full custom formatter based on numeral.js

Previous migration changes - removed now

## Migration changes

The tricky part here comes with the format migration for the new metric, as there's no easy way to detect what "Default" means, not having access to field formatters during migrations.
There are two possibilities, as far as I can see, on the topic:

when default format is detected, assume it's a number format with the compact feature turned on
when default format is detected, assume nothing and let the Kibana default format go on (no compact)

In this PR I've implemented the 2nd migration strategy.
Here's a visual example of the problem.

Original dashboard

Option 1

Option 2 (implemented)

Release notes

Default format in Lens will apply dataView field format as usual now in new Metric visualization.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

For maintainers

This was checked for breaking API changes and was labeled appropriately

…d-formatters

…atter

mohamedhamed-ahmed

Infra changes LGTM

## Summary In Cases, users can add lens visualizations in a case. The lens's attributes persist inside the `cases-comments` saved object. If the lens team adds a migration a migration will run for the `cases-comments` SOs to migrate any lens visualization inside the case comment. In this PR we defer the lens migrations to be done on read when it is fetched by users. This way, the cases comments will not be migrated at Kibana startup. Blocker for #158468 ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Kibana Machine <[email protected]>

…atter

kibana-ci · 2023-06-13T08:25:00Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 47d0211

Failed CI Steps

FTR Configs #25

Test Failures

[job] [logs] FTR Configs #25 / InfraOps App Metrics UI Hosts View #Single host Flyout "after each" hook for "should navigate to APM services after click"
[job] [logs] FTR Configs #25 / InfraOps App Metrics UI Hosts View #Single host Flyout should navigate to APM services after click

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`expressionMetricVis`	62	61	-1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/test-jest-helpers`	103	105	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`expressionMetricVis`	34.9KB	33.4KB	-1.5KB
`lens`	1.3MB	1.3MB	+1.7KB
total			+143.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`expressionMetricVis`	14.1KB	13.8KB	-276.0B
`lens`	35.2KB	35.3KB	+71.0B
total			-205.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/test-jest-helpers`	135	137	+2

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`expressionMetricVis`	1	0	-1
`securitySolution`	410	414	+4
total			+5

Total ESLint disabled count

id	before	after	diff
`enterpriseSearch`	20	22	+2
`expressionMetricVis`	1	0	-1
`securitySolution`	491	495	+4
total			+5

History

💚 Build #134220 succeeded 924c565
💔 Build #134090 failed 722001d
💔 Build #134040 failed 50a0ab3
💔 Build #133980 failed bb828ca
💔 Build #133933 failed ed751c3
💔 Build #133908 failed 12160c9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

wayneseymour · 2023-06-13T09:27:31Z

x-pack/plugins/lens/public/datasources/form_based/dimension_panel/dimension_panel.test.tsx

@@ -154,6 +157,18 @@ const bytesColumn: GenericIndexPatternColumn = {
  params: { format: { id: 'bytes' } },
 };

+const services = coreMock.createStart() as unknown as LensAppServices;


Marco says this would be a code smell in prod code, but this double type cast is ok in a test.

* main: [Lens] Add custom formatter within the Lens editor (elastic#158468) [Uptime] Hide app if no data is available (elastic#159118) [CodeEditor] Add grok highlighting (elastic#159334)

* main: (199 commits) [Lens] Add custom formatter within the Lens editor (elastic#158468) [Uptime] Hide app if no data is available (elastic#159118) [CodeEditor] Add grok highlighting (elastic#159334) Expose decoded cloudId components from the cloud plugin's contract (elastic#159442) [Profiling] Use collector and symbolizer integrations in the add data page (elastic#159481) [Infrastructure UI] Hosts View: Unified Search bar with auto-refresh enabled (elastic#157011) [APM] Add feature flag for not available apm schema (elastic#158911) [Lens] Remove deprecated componentWillReceiveProps usage (elastic#159502) [api-docs] 2023-06-13 Daily api_docs build (elastic#159536) [data views] Use versioned router for REST routes (elastic#158608) [maps] fix geo line source not loaded unless maps application is opened (elastic#159432) [Enterprise Search][Search application]Fix Create Api key url (elastic#159519) [Security Solution] Increase timeout for indexing hosts (elastic#159518) dashboard Reset button disable (elastic#159430) [Security Solution] Unskip Endpoint API tests after package fix (elastic#159484) [Synthetics] adjust alert timing (elastic#159511) [ResponseOps][rule registry] Remove usages of `refresh: true` (elastic#159252) Revert "Remove unused package (elastic#158597)" [Serverless] Adding config to disable authentication on task manager background worker utilization API (elastic#159505) Remove unused package (elastic#158597) ...

## Summary Part of [54306](#154306) Required #158468 to be merged first. This PR integrates only the migration part of the new Metric new formatter task. This migration implementation defines takes the simplest route for the migration route of metric visualization formatter: * if any custom formatter is set in Lens, then apply a `compact` flag on it to keep it consistent * if `Default` formatter is set in Lens, then assume nothing and let the regular formatter hierarchy do its own work ### Example Original dashboard ![image](https://github.com/elastic/kibana/assets/924948/e5cff808-456d-406e-a221-706c207f7629) After migration ![image](https://github.com/elastic/kibana/assets/924948/d3f2052f-13d8-4228-b446-0eaf5a339e13) ### Checklist Delete any items that are not applicable to this PR. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) - [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) ### Risk Matrix Delete this section if it is not applicable to this PR. Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release. When forming the risk matrix, consider some of the following examples and how they may potentially impact the change: | Risk | Probability | Severity | Mitigation/Notes | |---------------------------|-------------|----------|-------------------------| | Multiple Spaces—unexpected behavior in non-default Kibana Space. | Low | High | Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces. | | Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks. | High | Low | Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure. | | Code should gracefully handle cases when feature X or plugin Y are disabled. | Medium | High | Unit tests will verify that any feature flag or plugin combination still results in our service operational. | | [See more potential risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) | ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Stratoula Kalafateli <[email protected]>

dej611 added 21 commits May 16, 2023 18:11

🔧 Use kibana formatters

747adfe

Merge remote-tracking branch 'upstream/main' into feature/metric-fiel…

6034cc5

…d-formatters

✅ Fix unti tests + fix bug when transitioning with custom formatter

d286cd0

🐛 Use bytes formatter for bits

275cd71

🔧 Flag overriden formatters

caec5a9

♻️ Handle all combinations for formatters

0490805

✅ Update unit tests

bb93c97

Merge remote-tracking branch 'upstream/main' into feature/metric-fiel…

ea5cc8c

…d-formatters

✨ Add params for formatting method

0bdb11c

✅ Add transitioning test

9aeb0c2

✅ Fix existing tests for new metric

0634dc4

✅ Fix more functional tests

95844c4

✅ More tests fixed

2aefa3f

✅ Fix latest notation issues

a3f9c47

✅ Fix more tests

ab9a9fe

✅ Fix precision

d073437

✅ All fix should be addressed now

19a6255

:white_mark_check: One more fix

2b44254

♻️ Small refactor

a3dd0f8

Merge remote-tracking branch 'upstream/main' into feature/custom-form…

32e4608

…atter

✨ Add custom formatter + compact option

d623e87

dej611 added Team:Visualizations Visualization editors, elastic-charts and infrastructure Feature:Lens ci:cloud-deploy Create or update a Cloud deployment labels May 25, 2023

dej611 added 4 commits May 25, 2023 17:46

🔧 Fix id

a59b6e5

Merge remote-tracking branch 'upstream/main' into feature/custom-form…

57e526c

…atter

🔧 Add metric migration

75e47cb

🔧 Improve migration for metric formatters

08048ce

dej611 added the release_note:breaking label May 26, 2023

Merge branch 'main' into feature/custom-formatter

4fe2da5

dej611 added the v8.9.0 label Jun 9, 2023

dej611 added 2 commits June 9, 2023 10:02

Merge remote-tracking branch 'upstream/main' into feature/custom-form…

ac47d0b

…atter

⏪ revert changes

12160c9

cnasikas mentioned this pull request Jun 8, 2023

[Cases] Mark lens migrations as deferred #159291

Merged

2 tasks

dej611 added 2 commits June 9, 2023 11:16

⏪ Revert changes

ed751c3

✅ Update test

bb828ca

dej611 requested a review from a team as a code owner June 9, 2023 12:30

mohamedhamed-ahmed approved these changes Jun 9, 2023

View reviewed changes

dej611 and others added 2 commits June 9, 2023 16:29

✅ Round value

50a0ab3

✅ Fix precision

722001d

dej611 and others added 3 commits June 12, 2023 09:36

✅ Fix value

cf68fe0

Merge remote-tracking branch 'upstream/main' into feature/custom-form…

924c565

…atter

Merge branch 'main' into feature/custom-formatter

47d0211

This was referenced Jun 13, 2023

[Lens] Enable dataView field formatters in Metric visualization #157900

Closed

[Lens] Add new Metric formatter migrations #159539

Merged

wayneseymour reviewed Jun 13, 2023

View reviewed changes

dej611 merged commit 7b1d473 into elastic:main Jun 13, 2023

kibanamachine added the backport:skip This commit does not require backporting label Jun 13, 2023

dej611 mentioned this pull request Jun 13, 2023

[Lens] Enable dataview field formatters for the new metric #154306

Closed

dej611 mentioned this pull request Jun 13, 2023

[Lens] Support the duration formatter #153378

Closed

drewdaemon mentioned this pull request Sep 8, 2023

[O11y][Azure Metrics] Migrate Queue Storage Overview dashboard to lens elastic/integrations#7662

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Lens] Add custom formatter within the Lens editor #158468

[Lens] Add custom formatter within the Lens editor #158468

dej611 commented May 25, 2023 •

edited

Loading

mohamedhamed-ahmed left a comment

kibana-ci commented Jun 13, 2023

API count

ESLint disabled line counts

Total ESLint disabled count

wayneseymour Jun 13, 2023

[Lens] Add custom formatter within the Lens editor #158468

[Lens] Add custom formatter within the Lens editor #158468

Conversation

dej611 commented May 25, 2023 • edited Loading

Summary

Original dashboard

Option 1

Option 2 (implemented)

Release notes

Checklist

For maintainers

mohamedhamed-ahmed left a comment

Choose a reason for hiding this comment

kibana-ci commented Jun 13, 2023

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

ESLint disabled line counts

Total ESLint disabled count

History

wayneseymour Jun 13, 2023

Choose a reason for hiding this comment

dej611 commented May 25, 2023 •

edited

Loading