Service dependency view based on trace search criteria

[objectiser]

Requirement - what kind of business use case are you trying to solve?

The current service dependency tab presents service dependency information related to all services being traced based on information derived from a batch job that generally will only run once per day (Apache Spark job). User's have no ability to customise the view based on a subset of services related to specific trace paths, time range, tags, etc. As the information is only calculated at the end of day, it also means users have to wait until the following day to see the results - which means a significant delay in getting feedback on changes being deployed to their services.

The requirement is to provide a "service dependency view" as an alternative visualisation of the set of trace instances retrieved from the current trace overview page.

Problem - what in Jaeger blocks you from solving the requirement?

Currently users can only view service dependency information (in a production environment) if using the Apache Spark job - which generally will only run at the end of day, so there is a delay in seeing the results. The view also shows a graph of all services, so in an environment with a large number of microservices, this can be overwhelming.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Provide an alternative view from the trace overview page. So users can enter their search criteria and initially see the list of trace instances as now, and then switch view to see the same list of trace instances but now presented in a service dependency graph.

The graph can be constructed by analysing the client/server span pairs in each trace instance to extract the source and destination service names. Where a client span has no associated server span, it could use the peer.service tag or host from the http.url to provide a representation of the external service.

If we want to go beyond the current service dependency graph representation to provide more elaborate information:

• Relationships could be directional on the graph
• Latency information could be included, as well as number of requests
• Links could be per operation rather than just at the service level
• If a standard way of defining service version was supported, e.g. a version tag, then would be good to differentiate requests going to different versions of the service - as a way to identify when traffic has migrated away from an older version
• Auto refresh cycle

Some of these features could be enabled via checkboxes, to allow the initial view to be simple.

Any open questions to address

[objectiser]

This task has been discussed on recent bi-weekly community calls in relation to the deep dependency graph (DDG) work. The idea being that the DDG will initially obtain its information from the Jaeger backend, derived using a post processing (flink) job. However data source API could be defined in the UI to allow the DDG to be visualised based on the information obtained via a trace search.

On the last call (2019-07-26), it was agreed that "tasks (for alternate information source in UI - trace search results, and backend support for other storage options - ES) to be documented and referenced from" this issue.

cc @tiffon