Implement embeddable <LogStream /> component

[afgomez]

---

❗ Dear reader, if you come here looking for the component documentation, the up to date version is in the repo.

---

We want to make easy for other Kibana plugins to embed a simple log stream. That will enable other teams to show logs in whatever context they need (see #67611)

Internally it can also be used in some places of the logs UI, like the "View logs in context".

Goals

• Export a <LogStream /> component in a shared location.
• Show log lines between the specific startTimestamp and endTimestamp.
• Show log lines with a specific query.
• Handle positioning of the log stream with the center prop.
• Handle highlighted lines.

Non-goals (for MVP at least)

• Reuse this component in the Log stream page.
• Configure visible columns.
• Live stream.
• Context menu for the log lines.
• Pagination

Proposed API

On it's most basic form, the component takes a startTimestamp and endTimestamp parameters.

<LogStream
  startTimestamp={Date.now() - 86400000}
  endTimestamp={Date.now()}
/>

Querying data

The component accepts KQL expressions via the query prop. This allows consumers to show log lines in the context they need.

// APM traces
<LogStream
  startTimestamp={...}
  endTimestamp={...}
  query="trace.id: 12345"
/>

// Logs for a specific host
<LogStream
  startTimestamp={...}
  endTimestamp={...}
  query="host.name: foobar"
/>

Showing data

By default the component will show the newest entries. That is, the list will be scrolled all the way to the bottom.

Users can specify a middle point with the center prop. The center is a LogEntriesCursor.

<LogStream
  startTimestamp={...}
  endTimestamp={...}
  center={{time: ..., tiebreaker: ...}}
/>

The default page size is 200, and the component will not do pagination by default. To tweak this parameters, consumers can specify the paginate and pageSize props.

<LogStream
  startTimestamp={...}
  endTimestamp={...}
  paginate={true}
  pageSize={100}
/>

If the consumer wants to highlight a specific log line, it can do so by passing the log line ID to the highlight prop.

<LogStream
  startTimestamp={...}
  endTimestamp={...}
  highlight="abcde12345"
/>

Source configuration

The infra plugin has the concept of "source configuration" to store data about the logs UI, like what indices to query, or which columns to show.

The component will use this information to determine where and how to show the log data. By default it will use the default source configuration. Consumers of the component can specify a different source via props.

<LogStream
  startTimestamp={...}
  endTimestamp={...}
  sourceId="my_source"
/>

Implementation details

The component will rely in the existing /api/log_entries/entries API to fetch the log lines.

The component will reuse <ScrollableLogTextStreamView />. There is an example of how to use it outside the log stream in the View in context component.

The component will hold its own state. We will build new hooks for this to explore possible new ways to handle the state in the log stream page.

The log stream page won't use this component. Each will use their own mechanisms to hold state. Internally, each will use the same UI components and call the same backend APIs.

Related issues

• [APM] Display related trace logs in the trace sample content #67611
• [R&D] Logs embeddables: what are your use cases? #70513
• [Logs UI] Simple Logs component POC #72629

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[weltenwort]

This sounds great, thanks for spelling it out in detail. ❤️

The component API makes sense to me. I'd only prefer to call the source prop sourceId for clarity instead (so we keep the option of also allowing a full config object to be specified using a prop like sourceConfiguration in the future).

Nice job!

[weltenwort]

Another thought: Since you've invested the time to write this down, do we want to take the architecture description and put it into a markdown doc when we implement this? That way it might be more discoverable than in the description of a closed issue in the long term.

[afgomez]

Since you've invested the time to write this down, do we want to take the architecture description and put it into a markdown doc when we implement this?

Yes! I tried my best to write it as documentation for the consumers. We can copy-paste it either in a JSDoc on top of the component file or in a README.md in the same folder.

[chrisronline]

This looks great!

It sounds like the @elastic/stack-monitoring-ui team can replace our existing logs table with something like this!

cc @ravikesarwani

[sorantis]

+1 would love to integrate this in the enhanced host details.

cc: @hbharding