Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation to vis_augmenter #4527

Merged
merged 8 commits into from
Jul 14, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add documentation to vis_augmenter
Signed-off-by: Tyler Ohlsen <[email protected]>
ohltyler committed Jul 7, 2023
commit ea5b3536718cf8437300c45002856ca62039c60f
45 changes: 44 additions & 1 deletion src/plugins/vis_augmenter/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,44 @@
Contains interfaces, type definitions, helper functions, and services used for allowing external plugins to augment Visualizations. Registers the relevant saved object types and expression functions.
## Vis Augmenter

### Overview

The feature introduced in [#4361](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/4361) allows for plugins to be able to display additional data on a visualization. The framework for how that is done, including all of the interfaces, type definitions, helper functions, and services used, are maintained in this plugin. It also registers new saved object types and expression types that plugins use to standardize how data is formatted and visualized on the charts.

### Eligibility

Currently, this augmenting framework is only eligible for line visualizations, with a single date histogram x axis. There is future plans to expand this to other types of visualizations, as well as other ways that visualizations may be augmented, such as overlaying forecasting data.

### Framework
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is there a visual bock diagram that you can attach that outlines this framework? The description is nice but visuals will help understand the mental model better :)


The main idea is that plugins can hook into the render lifecycle of a visualization, such that when the original source datapoints are being fetched, plugins can also fetch data. All of this can then be combined and visualized cohesively on a chart. This process can be broken down into a few main steps:

1. Plugins implement a way to associate a plugin resource to a visualization. In the background, this means creating an `augment-vis` saved object linking the two.
2. Vis Augmenter will fetch any `augment-vis` objects that reference a particular visualization.
3. Vis Augmenter will run the plugin-registered expression function, along with any arguments, that is persisted in the saved object.
4. Vis Augmenter will combine all of the `VisLayer` results from the expression functions.
5. Vis Augmenter will update the source datatable with every `VisLayer`, dependent on the specific `VisLayer` type.
6. Visualization is rendered with the original data & augmented plugin data.

### `augment-vis` saved object

This is a new saved object type introduced with the primary purpose of maintaining an association of a plugin resource (e.g., anomaly detector, alerting monitor) to a visualization. It also contains the plugin-registered expression function that will be executed when fetching data for that particular resource, when rendering the visualization. The advantages of using saved objects to maintain these relationships is it keeps this relationship strictly a Dashboards-only feature, without coupling anything with the objects stored in OpenSearch. It also lets us use the concept of multitenancy to isolate these associations across different tenants.

### `VisLayer`s

`VisLayer` is the generic interface for a type of augmentation done on a visualization. The main idea is any plugin can hook into the rendering workflow of a visualization and have it fetch `VisLayer`s. This plugin will handle the processing of `VisLayer`s of different types, and how each type will be rendered consistently on a visualization. Currently, the only introduced one is `PointInTimeEventsVisLayer`. This will render a red triangle event datapoint below a source visualization, aligned with the visualization's temporal axis.

### View events flyout

This is the primary UI for providing a more detailed view on `VisLayer`s for a particular visualization. See the corresponding [README](src/plugins/vis_augmenter/public/view_events_flyout/README.md) for details.

### Steps for plugin integration

For an external plugin to utilize the Vis Augmenter plugin and its rendering framework, there are a few main steps needed:

1. Provide UX for handling associations between a visualization and plugin resources. There are examples in [Anomaly Detection](link) and [Alerting](link). This is where the bulk of the effort is needed. While there is no very strict requirements, there should be UX flows for creating, adding, and removing associated plugin resources to the visualization. Behind-the-scenes, this means making saved object API calls to handle CRUD operations of the `augment-vis` saved objects. Another important part is implementing & registering a `UIAction` so that there is a visible option under the visualization context menu to maintain associations. This is the current way that Anomaly Detection and Alerting inject their UX components into core Dashboards.

> It is worth calling out that currently there is no other registration being done by the plugins to potentially leverage common UX components or user flows, such as what's done in the Saved Objects Management plugin for different saved object types. There is absolutely improvements that can be done to decrease plugin responsibility and prevent duplicate code by persisting a lot of these common flows within Vis Augmenter. Given that this is an initial release with a lot of uncertainty on how the flows may differ across plugins, or what becomes the most important or common user flows, this was left as an open item for now.

2. Implement & register an expression function of type `vis_layers` that will fetch plugin data and format it into a `VisLayer`. This is what will be executed when a visualization is rendered and all of the chart's source data is being retrieved. It is important that the function does not simply return its own `VisLayer` result, but rather appends it to the list of input `VisLayer`s. By following this pattern, it allows an arbitrary group of plugins and arbitrary number of functions to be executed in any sequence, to all produce the same final set of `VisLayer`s for a particular visualization.

3. Define constant values to be included in the `augment-vis` saved objects. For example, all objects from a particular plugin should have a consistent `originPlugin`, `expressionFn`, and follow a consistent pattern for populating `pluginResource` values, such as URL generation.
14 changes: 14 additions & 0 deletions src/plugins/vis_augmenter/public/view_events_flyout/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## View Events Flyout

### Overview

This flyout provides a detailed view of all `VisLayer`s associated to a particular visualization. It consists of two main portions:

1. The top portion showing the base visualization, and the contextual time range that can be refreshed to fetch any new data in real time
2. The bottom portion showing a breakdown of each `VisLayer`, organized first by origin plugin (e.g., anomaly detection), and then by each plugin resource (e.g., anomaly detector). Each chart represents results produced only by the particular plugin resource. The resource name will also be a link, directing the user to the resource's details page within its respective plugin.

### Improvements

Currently, the charts rendered within this flyout are utilizing the `embeddables` plugin to re-create the source visualization, as well as data filtering and other visual changes, based on what is being rendered. Ideally, this can be decoupled from `embeddables` entirely, and the charts can all be rendered directly using expression renderers. This can help clean up and remove dependencies across plugins, as well as being able to remove the `VisAugmenterEmbeddableConfig` that has been added to `VisualizeEmbeddable` in order to persist some of the needed visual changes to the charts.

For more details on this, see the [GitHub issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/4483).