Add support to "archive traces"

[yurishkuro]

Users often ask for this feature. The trace retention period is usually short, and often people would put a link to the trace into some issue tracker to improve service performance, but by the time someone starts working on the ticket the trace has already expired.

The functionality is already supported in the backend, but missing UI changes. The idea is to add a button "Archive Trace" on the trace detail view (next to View Options dropdown), which opens (expands?) a form with:

• a dropdown to select the TTL (options: 1 month, 3 month, 1yr, indefinite)
• submit button that does a POST to /api/v1/archive/{trace_id}?ttldays=30 (or whichever is the best REST API, maybe move trace_id/ttldays to POST body)
• once the trace is archived, the form displays the permanent URL: http://.../archive/{trace_id}

The /archive/ route needs to make the webapp query /api/v1/archive/{trace_id} endpoint instead of the usual /traces endpoint. And it must also change View JSON to the same URL.

@saminzadeh wdyt? how hard is it to do?

[saminzadeh]

Will archived traces should up in the trace search results or will you have to directly access them?

Is there a way we can get around building the endpoint without having the UI know about a /archive/trace endpoint? It seems like that's an implementation detail of the backend/storage layer.

e.g. If we have Trace ID = 1 and Trace ID = 2 and 1 was archived, we should still be able to pull the trace via GET /traces/1. The backend could abstract this out and check the archive cache if it exists before returning a 200 vs 404. We can add a meta property to the trace which gives its current TTL and expiry information.

{
  spans: [{...}},
  processes: {...}
  meta: {
    ttl_days: 2
    expires: "2017-04-21T12:28:10-04:00"
  }
  ...
}

[yurishkuro]

I am open to opinions, but my preference was to have archive under separate URL, so that people know that the trace will expire if they don't archive. Having everything served from /traces/ obscures that distinction. But I don't have a super strong feeling about it.

[black-adder]

I prefer shan's suggestion.

[yurishkuro]

The backend changes have been implemented. We need a simplified version of the UI workflow:

• add Archive Trace button to the trace view
• execute a POST request against /api/archive/{trace-id}
• on success, replace Archive Trace button with archived indicator
• archived trace will be accessible at the same URL as before (backend falls back on archive storage if primary storage does not find the trace)

[tiffon]

@yurishkuro Does the backend return the traces with the meta: { ttl_days: .... } @saminzadeh mentioned?

[yurishkuro]

no

[Dieterbe]

what is the actual high-level use/case or desired purpose for archiving?

[yurishkuro]

jaegertracing/jaeger#57

[yurishkuro]

@tiffon this is now supported in the oss version of the query service.

Can we add a UI widget to the Trace View to do a POST to /api/archive/{trace_id}?

[tiffon]

@yurishkuro Will do.

Is there a way to tell if a trace being viewed is an archived trace?

[yurishkuro]

No, there is no way. But (a) this only works with Cassandra, and (b) you can just delete the trace from the main keyspace.

Having said that, it's not that difficult to add a note to the trace that it was loaded from archive storage. Maybe as trace.warnings? I don't recall if we actually serialize those into json.

[tiffon]

@yurishkuro Sounds like we should include a toggle for enabling this in the query-config discussed in #42. It can be enabled when the query-service is using Cassandra.

.warnings is included in the trace payload. Adding a flag there would be great.

I don't follow (b). Seems like that shouldn't be a concern for the UI?

[yurishkuro]

I don't follow (b). Seems like that shouldn't be a concern for the UI?

I mean for testing purposes, you can do that. In production, I don't think it's that critical to display if the trace was loaded from archive, weak nice to have.

[tiffon]

@yurishkuro It'd be nice to provide some feedback to the user indicating the trace was successfully archived. Otherwise, it will just seem like nothing happened.

[yurishkuro]

the POST request returns success/failure, so you can provide feedback at the time of user asking to archive the trace.

[pavolloffay]

@tiffon is this done?

[yurishkuro]

we have it working internally. It requires "archiveEnabled": true in the ui config, and configuring the archive storage in jaeger-query.

[pavolloffay]

From the UI perspective the only missing part is to add docs to https://www.jaegertracing.io/docs/1.7/deployment/