[REQUEST]: APM agent direct logs

[bmorelli25]

Description

Document APM agents sending logs directly to APM Server.

Resources

• Add Logs Intake API (POC) apm-server#8757
• Ship logs via intake API apm-agent-java#2694

Collaboration

TBD. The docs and product team will work together to determine the best path forward.

Point of contact.

Main contact: @SylvainJuge

Stakeholders: @felixbarny

[felixbarny]

As there are multiple ways to send logs now (Filebeat, Elastic Agent, APM Agents), we should give an overview of the options, their tradeoffs, and provide guidance on when to use which option.

Also, in it's current form, the APM Agents option has a few caveats that should be documented. For example, custom fields aren't supported and a high rate in logs may result in transactions and metrics to be dropped.

[bmorelli25]

Step one complete in #2452. A list of follow-up updates is in the description.

[SylvainJuge]

@bmorelli25 the support for direct logs sending has been merged for Java APM agent (thus about to be released soon). Is there anything I could contribute here to move this forward ?

[bmorelli25]

Thanks for the nudge, @SylvainJuge :)

I've opened #2542 and elastic/ecs-logging#72 to address the follow-ups.

The only thing left is to update links to the Java agent docs after they are live. I didn't have time today to open a draft PR for that, but I will tomorrow.

[felixbarny]

I think we should also link to the Java agent docs for directly sending logs from the ecs-logging-java docs

[bmorelli25]

Everything is done except for links to the Java agent docs from the logs guide and the ECS logging book. Please leave a comment here when direct logs has been released, and I'll publish those last two links.

[SylvainJuge]

@bmorelli25 the Java agent 1.36.0 was just released a few minutes ago, thus log sending is now official.

[bmorelli25]

All done 🚢