Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM][OTel] Ensure Errors views works with OTel Data #9

Closed
wants to merge 5 commits into from
Closed

[APM][OTel] Ensure Errors views works with OTel Data #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f9ccf34
refactor: improve get_error_group_main_statistics
rmyz Oct 4, 2024
71cfaad
refactor: add stack_trace mapping
rmyz Oct 4, 2024
e50cf41
refactor: adapt errors to use fields instead of _source
rmyz Oct 4, 2024
4a11176
test: adapt snapshot
rmyz Oct 4, 2024
ee8ceda
refactor: remove useless optional chaining
rmyz Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/kbn-apm-types/src/es_fields/apm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ export const ERROR_EXC_MESSAGE = 'error.exception.message'; // only to be used i
export const ERROR_EXC_HANDLED = 'error.exception.handled'; // only to be used in es queries, since error.exception is now an array
export const ERROR_EXC_TYPE = 'error.exception.type';
export const ERROR_PAGE_URL = 'error.page.url';
export const ERROR_STACK_TRACE = 'error.stack_trace';
export const ERROR_TYPE = 'error.type';

// METRICS
Expand Down
6 changes: 6 additions & 0 deletions .../plugins/observability_solution/apm/common/es_fields/__snapshots__/es_fields.test.ts.snap
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 11 additions & 7 deletions ...ity_solution/apm/server/routes/errors/get_error_groups/get_error_group_main_statistics.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

import { AggregationsAggregateOrder } from '@elastic/elasticsearch/lib/api/types';
import { kqlQuery, rangeQuery, termQuery, wildcardQuery } from '@kbn/observability-plugin/server';
import { APMError } from '@kbn/apm-types';
import { errorGroupMainStatisticsMapping } from '../../../utils/es_fields_mappings/error';
import {
ERROR_CULPRIT,
ERROR_EXC_HANDLED,
Expand Down Expand Up @@ -129,7 +131,7 @@ export async function getErrorGroupMainStatistics({
sample: {
top_hits: {
size: 1,
_source: [
fields: [
TRACE_ID,
ERROR_LOG_MESSAGE,
ERROR_EXC_MESSAGE,
Expand Down Expand Up @@ -157,15 +159,17 @@ export async function getErrorGroupMainStatistics({

const errorGroups =
response.aggregations?.error_groups.buckets.map((bucket) => {
const normalizedFields = errorGroupMainStatisticsMapping(bucket.sample.hits.hits[0].fields);

return {
groupId: bucket.key as string,
name: getErrorName(bucket.sample.hits.hits[0]._source),
lastSeen: new Date(bucket.sample.hits.hits[0]._source['@timestamp']).getTime(),
name: getErrorName({ error: normalizedFields?.error } as APMError),
lastSeen: new Date(normalizedFields?.['@timestamp'] as string).getTime(),
occurrences: bucket.doc_count,
culprit: bucket.sample.hits.hits[0]._source.error.culprit,
handled: bucket.sample.hits.hits[0]._source.error.exception?.[0].handled,
type: bucket.sample.hits.hits[0]._source.error.exception?.[0].type,
traceId: bucket.sample.hits.hits[0]._source.trace?.id,
culprit: normalizedFields?.error.culprit,
handled: normalizedFields?.error.exception[0].handled,
type: normalizedFields?.error.exception[0].type,
traceId: normalizedFields?.trace.id,
};
}) ?? [];

Expand Down
9 changes: 6 additions & 3 deletions ...vability_solution/apm/server/routes/errors/get_error_groups/get_error_group_sample_ids.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
*/

import { rangeQuery, kqlQuery } from '@kbn/observability-plugin/server';
import { normalizeValue } from '../../../utils/es_fields_mappings/es_fields_mappings_helpers';
import { asMutableArray } from '../../../../common/utils/as_mutable_array';
import {
ERROR_GROUP_ID,
Expand Down Expand Up @@ -66,16 +67,18 @@ export async function getErrorGroupSampleIds({
should: [{ term: { [TRANSACTION_SAMPLED]: true } }], // prefer error samples with related transactions
},
},
_source: [ERROR_ID, 'transaction'],
fields: [ERROR_ID],
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken, the 'transaction' was added for the scoring, so I think the field TRANSACTION_SAMPLED, should we also added not sure why just 'transaction' was in this case
@dgieselaar can you confirm?

sort: asMutableArray([
{ _score: { order: 'desc' } }, // sort by _score first to ensure that errors with transaction.sampled:true ends up on top
{ '@timestamp': { order: 'desc' } }, // sort by timestamp to get the most recent error
] as const),
},
});
const errorSampleIds = resp.hits.hits.map((item) => {
const source = item._source;
return source?.error?.id;
const fields = item.fields;
const errorId = normalizeValue<string>(fields?.['error.id']);

return errorId;
});

return {
Expand Down
10 changes: 6 additions & 4 deletions ...ervability_solution/apm/server/routes/errors/get_error_groups/get_error_sample_details.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { APMEventClient } from '../../../lib/helpers/create_es_client/create_apm
import { getTransaction } from '../../transactions/get_transaction';
import { Transaction } from '../../../../typings/es_schemas/ui/transaction';
import { APMError } from '../../../../typings/es_schemas/ui/apm_error';
import { errorSampleDetailsMapping } from '../../../utils/es_fields_mappings/error';

export interface ErrorSampleDetailsResponse {
transaction: Transaction | undefined;
Expand Down Expand Up @@ -60,24 +61,25 @@ export async function getErrorSampleDetails({
],
},
},
fields: ['*'],
},
};

const resp = await apmEventClient.search('get_error_sample_details', params);

const error = resp.hits.hits[0]?._source;
const error = errorSampleDetailsMapping(resp.hits.hits[0]?.fields) as APMError;
const transactionId = error?.transaction?.id;
const traceId = error?.trace?.id;

let transaction;
let transaction: Transaction | undefined;
if (transactionId && traceId) {
transaction = await getTransaction({
transaction = (await getTransaction({
transactionId,
traceId,
apmEventClient,
start,
end,
});
})) as Transaction;
}

return {
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/observability_solution/apm/server/utils/es_fields_mappings/error.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
ERROR_EXC_HANDLED,
ERROR_EXC_TYPE,
ERROR_CULPRIT,
ERROR_STACK_TRACE,
AT_TIMESTAMP,
APMError,
OBSERVER_TYPE,
Expand Down Expand Up @@ -73,7 +74,6 @@ export const errorGroupMainStatisticsMapping = (fields: Fields) => {
id: normalizeValue<string>(fields[TRACE_ID]),
},
error: {
id: normalizeValue<string>(fields[ERROR_ID]),
log: {
message: normalizeValue<string>(fields[ERROR_LOG_MESSAGE]),
},
Expand All @@ -85,7 +85,6 @@ export const errorGroupMainStatisticsMapping = (fields: Fields) => {
},
],
culprit: normalizeValue<string>(fields[ERROR_CULPRIT]),
grouping_key: normalizeValue<string>(fields[ERROR_GROUP_ID]),
},
'@timestamp': normalizeValue<string>(fields[AT_TIMESTAMP]),
};
Expand Down Expand Up @@ -121,6 +120,7 @@ export const errorSampleDetailsMapping = (fields: Fields): APMError | undefined
},
],
grouping_key: normalizeValue<string>(fields[ERROR_GROUP_ID]),
stack_trace: normalizeValue<string>(fields[ERROR_STACK_TRACE]),
},
processor: {
name: normalizeValue<'error'>(fields[PROCESSOR_NAME]),
Expand Down