Custom drilldown links for a dashboard panel

[alexfrancoeur]

Why this is important

The ability to drill down from one dashboard to another view creates a custom workflow for troubleshooting and analyzing data. When speaking with experienced users, this is a fairly common ask. When speaking with new users, the expectation is that the behavior is already available within Kibana.

What we have in mind

In the definition of a dashboard panel, we will allow a user to define one or multiple custom drilldown links that can be easily accessed from that dashboard panel. Clicking into this drilldown link will navigate users to the preconfigured view.

Example custom workflows

• Dashboard for overall status of multiple data centers ➡️ Drill down to dashboard for a single data center ➡️ Drill down to a dashboard of a single server
• Dashboard with visualization for overall HTTP responses ➡️ Drill down to saved search for HTTP responses with filters from previous dashboard
• Operational dashboard panel ➡️ drill down to internal ticketing system

Proposed approach

Users should be able to navigate to the following:

• Kibana saved objects
  • Dashboard
  • Visualization
  • Discover saved search
• Absolute URLs
  • Any internal or external URL a user defines

For each drilldown navigation configured, each should have the following options:

• Kibana saved objects
  • Option to inherit state from previous dashboard. This would include filters, query, timeframe, etc. This setting would be enabled by default but if not selected, we'd use the saved objects originally configured state
  • Allow for additional custom static filters, query or timeframe that are specific to that drilldown link. This would allow for re-use of saved objects with filter's explicitly applied only for that drilldown link
• All drilldown links
  • Configurable navigation - direct navigation (default), open in new tab, open in new window

Initial concept

• Configuration would be done per panel and the drilldown links will be stored with the dashboard panel, not the visualization. This will allow for re-use of visualizations on multiple dashboards with different drilldown links.
• Links will be seen only on hover as to not affect reporting or visualization views
• A user can add as many links to a panel as they'd like
• Some future concepts that may compliment this new capability include a "Copy panel to other dashboard" option and a standalone "navigation panel" for links defined in that dashboard.

Related issues and PR's

• Drill down from one visualization to another
• Improve workflow for drill-down from chart to individual documents
• Add a click handler for metric visualization

cc: @stacey-gammon @snide @thomasneirynck @kobelb

[trevan]

It would also be nice to be able to drill down and add a dynamic filter based on a click. This would be for where I have a table/line that shows multiple servers and high level values. I'd like to click on the server and then go to a different dashboard to see details for that specific server. Looking at your mockups and description, it sounds like I would need to first click on the table/line for that server to add it as a filter, wait for the query to load (since the visualization panel will gray out), then click on these new links to drill down.

Also, it kind of fixes the click handler metric visualization but not entirely since that issue is more about getting metric visualization on par with the rest of the visualizations where you can add a filter based on the data in the visualization. These links might get it part of the way but it will still be kind of weird since most visualizations allow clicking on the data to filter while metric doesn't.

[snide]

Here's another design, but for what a "Dashboard link menu" would look like. Essentially same UI as the panel drilldowns in the above design, but rather than having a visualization, the view mode for this is just a list of exposed links. Figured I'd through it in here since the concepts are so similar and will likely reuse a lot of the same code.

[source-ram]

+1

[alexfrancoeur]

@trevan you are correct, in the proposed approach above you would need to click into a filter and then navigate to the URL. This implementation requires the least amount of change to the current users workflow. However, I think both approaches could be beneficial as having links that are unrelated to a filter may not necessarily need to be accessed via a data point. From a user experience perspective, what would your expectation be? Some thoughts below

• Click into a data point
  • If one or multiple drilldowns defined, show tooltip ➡️ options include filter entire dashboard or navigate to new view(s) / url(s) with filter applied
• If no drilldowns defined, don't show tools tip ➡️ filter entire dashboard
• Click into a data point
  • If drilldown defined, immediately navigate to new view with filter applied
  • If no drilldown defined, apply filter globally

Regarding the metric visualization. If you were able to define a link with pre-defined filters, you'd technically be able to provide one or multiple drilldowns with that metric pre-filtered. However, I agree - it's not exactly the same as clicking into the metric to navigate.

[thekofimensah]

I think this is a great idea, especially because most users need to integrate kibana with several other websites/programs that their system is connected to. +1

[RomainXie]

We need this feature just yet.
Suggest to add a option in the dashboard design, allow a internal URL which can transfer the query string & filter to the next automatically.

[stacey-gammon]

@snide - as I was implementing the mocks I came across a situation that might crop up with the way we do the panel flipping for the editing screen.

When the panel is really small, the link editing space will be really small too, and it's a little weird to make the user expand it just to edit links, and then have to resize it small again so the dashboard goes back to the way they want it.

I like the panel flipping idea, but this might be a blocker. Sounds like our other option is just using a modal?

Here are a couple screenshots that show my concern. Obviously the contents of the add/edit links panel will be a lot nicer, but you can get the idea that it will all be very squished.

I just thought of something else. The location of the link icon that shows up on hover is in the same spot as the location of the spy toggle. The spy toggle is internal to the visualization - it's not added by dashboard. So if we want to add a clickable panel icon on top of the actual visualization space, we can't make any assumptions about what it is hovering over (anyone can write anything to be an embeddable and put stuff there - so just moving the spy toggle may not be a sufficient solution).

We could add a footer to the panel that doesn't overlap the vis, but people might be unhappy with the added space it takes up. We could have it pop out on hover, below the visualization. Perhaps if we do that and make it "floatable" it'll work - shouldn't matter if the footer floats over a visualization below it, because the mouse is in the top visualization so the user isn't possibly trying to click something and can't get to it.

[trevan]

@stacey-gammon, is it going to be possible for the same visualization to have different "links" depending on the dashboard it is in? If not, why not leave the editing of the links in the visualization editor? It feels weird to edit a visualization specific item inside of the dashboard.

Also, why not put the "link icon" in the next to the "expand panel" icon? Or always show the dropdown in the top left that shows up in edit mode and have 'expand panel' and 'links' be in that menu? You could even move the spy link there as well.

And are these links going to be dynamic per my comments above? Will I be able to click/hover/interact on a row/bar/line/etc and get a link that includes the data that I'm interacting with?

[stacey-gammon]

@trevan

is it going to be possible for the same visualization to have different "links" depending on the dashboard it is in?
Yes

If not, why not leave the editing of the links in the visualization editor? It feels weird to edit a visualization specific item inside of the dashboard.
Another reason we went this route is because we wanted it to be a feature that any "embeddable", not just visualizations. For instance, we have visualizations and saved searches. Without making it part of the dashboard panel, you'd have to implement the same logic in a visualization and a saved search.

This topic of a pluggable dashboard embeddable vs a pluggable visualization type is somewhat complicated (and probably off topic for this specific feature). We have gone back and forth on whether everything should be implemented as a visualization plugin, or whether we should allow a more flexible "embeddable" plugin. We went with the later, though jury is still out on whether that's the best route. For now, with all the refactoring our visualize code was going through, it was easier to extract saved search logic out of dashboard code with a new embeddable plugin rather than implement it as a visualization plugin.

Also, why not put the "link icon" in the next to the "expand panel" icon? Or always show the dropdown in the top left that shows up in edit mode and have 'expand panel' and 'links' be in that menu? You could even move the spy link there as well.

I think this is a good option! Spy toggle is part of the embeddable so dashboard doesn't know anything about it. We could consider moving it out but it not sure we want to assume every embeddable will have a spy option.

And are these links going to be dynamic per my comments above? Will I be able to click/hover/interact on a row/bar/line/etc and get a link that includes the data that I'm interacting with?
No. You create the links up front, but after that they are static. Having the links be part of the hovering options on a visualization would be a visualization specific feature. I think this would be a useful next step, but we'd have to reconcile it with how we do filtering now.

So I think for your use case, you probably want visualization specific drill down links that are customizable on the visualization itself, and are specific to the type of visualization shown, so clicking might not filter, but would open up a new object. Perhaps we should have two, separate issues to cover both those use cases.

[trevan]

@stacey-gammon, yeah I've been following the external discussions about embedded options. I also re-read the description here and see where it said this should be stored on the dashboard state instead of the visualization state.

So I think for your use case, you probably want visualization specific drill down links that are customizable on the visualization itself, and are specific to the type of visualization shown, so clicking might not filter, but would open up a new object. Perhaps we should have two, separate issues to cover both those use cases.

They don't have to be visualization specific drill down links. If you look at the "related issues" in the description, almost all of them talk about clicking on something in the visualization/search/etc that then takes them to a different dashboard/visualization/search filtered based off of what they just clicked on. These aren't links that you define before hand but links that are dynamic based off of the data currently shown in the visualization/search.

[stacey-gammon]

@trevan

They don't have to be visualization specific drill down links.

I don't know if I follow this. What I mean to say is that those types of links would have to be configured during creation of the visualization. For instance your request to make a data table cell clickable to a dynamic link by inserting the value you clicked into a url template - that is something that would be very specific to data tables, and not applicable for a bar chart.

I've thought a bit about whether we could generalize this somehow at the dashboard level by having embeddables send their filtering through dashboard, which could then intercept it and rather than add a filter, it would redirect the user to a link, stuffing the filter into a url template defined at the panel level. I'm not convinced this is the right solution though because it requires more coupling of the embeddable to the dashboard, and it'd still be possible for someone to write an embeddable and apply the filter directly.

@snide
Some more questions regarding the UI/UX flow:

• What happens if I click Add link more than once? Do I get additional input lines, or should I be forced to finish, or delete the one I just started first?
• The saved object list might be too simple. I think we need to distinguish between types (saved searches, visualizations and dashboards). Kind of like the listing pages, but showing all saved objects (except some I suppose - probably not index patterns). Also brings up a question of whether we should show saved objects added by plugins. Probably, but we need for all of them to be able to give us a way to construct a navigation url. I'm thinking the control might be too small for all our needs? We could add in a listing table minus the check boxes, but not sure how that would fit in the pop up.
• After I click Add link but before I select a saved object, should we give the user a way to cancel and remove the newly added link? Otherwise I think they have to pick an object then remove it via the edit tool? What if there are no objects to pick, how would a user remove the line then?

[trevan]

@stacey-gammon

They don't have to be visualization specific drill down links.

I don't know if I follow this. What I mean to say is that those types of links would have to be configured during creation of the visualization. For instance your request to make a data table cell clickable to a dynamic link by inserting the value you clicked into a url template - that is something that would be very specific to data tables, and not applicable for a bar chart.

I was just trying to say that if links will be per visualization per dashbard (which is what I gathered from your response), then these "dynamic" links also can be per visualization per dashboard. You would just edit them in the dashboard and then a visualization can have different dynamic links depending on the dashboard it is in. That is what it sounds like is going to happen for the "static" links that you are currently designing.

I've thought a bit about whether we could generalize this somehow at the dashboard level by having embeddables send their filtering through dashboard, which could then intercept it and rather than add a filter, it would redirect the user to a link, stuffing the filter into a url template defined at the panel level. I'm not convinced this is the right solution though because it requires more coupling of the embeddable to the dashboard, and it'd still be possible for someone to write an embeddable and apply the filter directly.

I've actually thought about adding that "intercept" ability so that plugins can hook into it and do exactly what you said.

What would really be nice is that instead of causing a filter when you click on an item (either it is a visualize table row, a search table row, a visualize bar chart, or a different embeddable thing), a context menu appears. This context menu then gives you the option of "filtering" (with both a positive and negative option), go to a link, create watch, create ml job, etc (all of these actions are defined by the embeddable or by plugins). Dashboard creators would then specify what the link templates should look like (with, preferably, easy shortcuts to other dashboards/visualizations/etc) and on click, the embeddable would request the app state and fill in the link.

[stacey-gammon]

Good thoughts @trevan, I'd like to explore them more. The static links will probably remain a first step, but it sounds like most use cases will require the dynamic filtering context menu. btw, we do have plans for a "create ml job" link in the panel, so will need to expose more pluggability in the dashboard panel anyhow.

[stacey-gammon]

btw @trevan - it should technically be the same amount of clicks to accomplish the same goal. E.g.

Looking at your mockups and description, it sounds like I would need to first click on the table/line for that server to add it as a filter, wait for the query to load (since the visualization panel will gray out), then click on these new links to drill down.

Because the links will be on the panel, not the embeddable, you should be able to click the drill down links before the embeddable finishes rendering. Same with if you built a "Dashboard link menu" - that won't require any network requests so the panel should finish loading nearly instantly and then dashboard level drill down links will be available immediately after a filter is clicked and applied at the dashboard level.

It might be a little bit of a clunkier UI/UX than your suggestion, but it does retain the benefit of one click filtering for people that don't want to use drill down links.

[trevan]

@stacey-gammon, that is definitely clunky and isn't very intuitive (how do you tell people to make sure they add a filter before they click on the link?). You also still have the issue of sending a request to ES to rebuild the dashboard that you are just going to throw away. That increases the load on ES for little benefit.

I don't understand what you mean by the "Dashboard link menu".

[ghost]

For REF:

Its called "dynamic drill-downs" in Splunk. We used advanced XML , leveraging tokens within the code of the app. The next piece that is relative and missing here is called "workflow actions" in Splunk - being able to pull token data from the page and use it in a workflow URL - IE: passing to an external REST API..
Workflow actions has its own admin UI...

Cheers!

[alexfrancoeur]

thanks for the additional info @RANGERBEE! We've taken a step back and are hoping to approach this in a dynamic sense as most users here have requested. I plan on putting together more detailed specs with mocks in the coming weeks. Will update this issue when ready.

[ghost]

no problem... To really complete the workflow and surpass Splunk, making available a customizable secure landing page for each Workflow action genre that allows for some JS would be seriously trick! When we get those REST API response headers they need to be handled, the user does not need to see it, so a landing page with an iframe was our solution to both handle the request/responses and forward to a proper URL then after... ;)

[Braj-120]

Hi @alexfrancoeur,
Any timeline on this feature? It would be a really nice feature to have.

[alexfrancoeur]

@Braj-120 Unfortunately we do not have a specific timeframe at the moment but we are hoping to deliver in 6.x. I'll be updating the original description of this issue in the coming days to reflect our most recent requirements.

[Autorola-LHH]

Also hoping to use functionality like this. Being able to go directly from a dashboard with visualizations to the underlying searches would make troubleshooting so much easier.

[vaibhavrodage]

+1

[AndrewMcQuerry]

+1

[warnerrj79]

+1 from me. This feature would be brilliant to have

[cheriemilk]

Hi alexfrancoeur,

when this enhancement cab be released?
There's a user scenario from me(please check below). Does it include in this requirement?

I have data like below in Kibana and created a pie chart based on field 'ktf2'. In the pie chart, I want to view corresponding 'cn' information by clicking one of the slices of the pie chart.

cn,ci,at,ktf2
Siemens, SiemensPS, Search, "Competency_Criteria,Profile_Criteria"
HSBC, HSBCPROD, Search, "Keyword, Competency_Criteria, Background_Criteria"
......

Thanks,
Cherie

[mashuai191]

+1

[damianward1972]

Let's do this, please... without it, Kibana is pretty limited. My users are desperate for this type of functionality +1

[zenglime]

There is a lot of stuff discussed in this issue that would be really useful. However I wanted to point out to those waiting for these features, that there are things that can be done in Kibana today to dynamically link dashboards and facilitate a number of drill-down scenarios. The relevant features might not be obvious, and it does involve taking some time to figure out the structure of a Kibana URL, including how to set filters by manipulating the URL directly. While this requires a bit of work, the results can be very useful. Let me share a few examples...

Use the markdown visualization to group related dashboards

The markdown visualization can be used to navigate between dashboards, which is especially useful for dashboards related to the same use-case.

This is achieved using simple hyperlinks to the dashboards:

TIP: We recommend modifying the Kibana advanced setting filters:pinnedByDefault, setting it to true. This will keep filters set on one dashboard, when navigating to another. This is very useful when filtering data to drill-down and then changing dashboards for a different perspective on that same data.

Use the Markdown feature of Visual Builder

1. Define the data.

1. Use the data to dynamically set filters via the URL that is used in the Markdown links (once you have figured out the URL structure it is not as hard as looks here).

1. Set any relevant filters and even use CSS to tweak how stuff is displayed.

1. The result

Clicking on the color goes to one dashboard...

And clicking on the number links to another...

Notice how the filters are set via the URL. This is how we make dashboards to launch into certain devices, applications and use-cases, like these...

Use the URL field formatter

1. Goto the Index Pattern configuration within Kibana, and edit the settings for the field that you would like to use within a URL. The format will be default:

1. Change it to Url.
2. Ensure the type is Link
3. Label Template should be set to {{value}}
4. The Url Template should include the URL to open with {{value}} inserted where the field value will be.
5. You can also select whether to launch the URL in a new tab or not.

Together this looks like this...

If you use these fields in a visualization such as a table, you will notice the values are now hyperlinks.

Clicking on them will launch the configured URL. In the example above the Talos page is launched for that IP address.

This same method can be used to launch to another dashboard, leveraging the Kibana URL methods showed in the Markdown example...

While these techniques do not handle all of the scenarios that the things discussed in this issue hopefully will. They can be used to significantly improve users' interactions with Kibana.

We use these techniques to allow the users of our Network and Security Analytics solution to use Kibana without ever having to open the full list of dashboards. The navigation and drill-down options we provide allows them to navigate to any perspective they need, and to rarely need to manually set filters when doing so.

Hopefully some of you find these tips helpful as you wait for future enhancements.

very cool

[pgege]

At GoodLabs Studio, one of the most asked-for features we get from our clients that use Kibana is the drill-down feature. This thread was started 3 years ago and not much has changed, so we decided to take a crack at it and open source it for others to use.

Our drill-down plugin allows users to access as many targets as they want; targets can range from other Kibana dashboards or visualizations to external URLs. All of which are presented in a simple popup. In addition the targets can be configured to include filters that are applied on the next view. Users configure the drill-down feature the same way they configure index patterns by simply applying the drill-down field formatter provided by our plugin to their data. The drill-down field formatter can be applied to multiple data types and as discussed above, there are no limits to the number of drill-down targets that can be added.

Filters are applied through URL templates that the user configures when they setup the drill-down field formatter. Within this post thread, one of the posts states - “add a dynamic filter based on a click. This would be for where I have a table/line that shows multiple servers and high level values. I'd like to click on the server and then go to a different dashboard to see details for that specific server.” This post highlights exactly what we have implemented, this feature allows users to analyze their data with less actions—making kibana easier to use. For those that are interested, the plugin can be found at: https://github.com/goodlabs-studio/kibana-plugin-drilldownmenu. Any feedback is welcome!

[AlonaNadler]

Few updates on this issue. Few weeks Kibana team merged drilldown into master. You can now create drilldowns from one dashboard to the other configuring what you wish to carry over to the desitnation (global filters, search query, and timerange). For each panel you can have multiple drilldowns for different destinations. On interaction users will have options to choose where to go (or if to perform the regular filter).If all goes well that will be part of the upcoming release. Please let us know if you find it useful and if anything is missing

In addition we already are working on the next steps with different more advanced drilldown including to external urls and other solutions within Kibana #63710
Thanks you for continuing providing feedback 👍

cc: @elastic-jb @streamich @Dosant

[FireEmerald]

@AlonaNadler Is there any easy way (like a docker image) to try the merged drilldown feature?

[timroes]

Closing this as drilldowns has been released in Kibana.

[dalzzing2]

@timroes
From which version can this feature be available?
I am using version 6.7.1, is this possible?

[Dosant]

@dalzzing2, dashboard drilldowns released in 7.8 and then got a bunch of improvements in later versions.
Current documentation: https://www.elastic.co/guide/en/kibana/current/drilldowns.html
There was also a webinar: https://www.elastic.co/webinars/dashboard-drilldowns-with-data-visualizations-in-kibana