[8.14](backport #38941) Migrate X-PACK Filebeat pipeline to main Beat pipeline

[mergify]

Proposed commit message

Migrating the x-pack filebeat pipeline into the centralized Beats pipelines

Testing

• Simple build "default" case
  https://buildkite.com/elastic/beats/builds/5731#018ee6f4-2d18-4a97-882b-2aa0d90dfa65

• Adding all the labels
  https://buildkite.com/elastic/beats/builds/5764

• Running cloud tests build
  https://buildkite.com/elastic/beats-xpack-filebeat/builds/1655#018eeb54-b690-431f-8d4b-59cd33a9870f
  Same build from Jenkins https://beats-ci.elastic.co/blue/organizations/jenkins/Beats%2Fbeats/detail/PR-38941/16/pipeline/2687/

Signed-off-by: Alexandros Sapranidis [email protected]

---

This is an automatic backport of pull request #38941 done by [Mergify](https://mergify.com).

[botelastic]

This pull request doesn't have a Team:<team> label.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2024-04-18T14:30:07.700+0000

• Duration: 15 min 41 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3
Skipped	0
Total	3

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.14/pr-38941 upstream/mergify/bp/8.14/pr-38941
git merge upstream/8.14
git push upstream mergify/bp/8.14/pr-38941

[dliappis]

This PR is re-introducing xpack/heartbeat in 8.14, see

beats/.buildkite/scripts/common.sh

Line 26 in 3918ab8

[ -z "${run_xpack_libbeat_arm_tests+x}" ] && run_xpack_libbeat_arm_tests="$(buildkite-agent meta-data get run_xpack_libbeat_arm_tests --default "false")"

for reference

[dliappis]

LGTM

[dliappis]

For https://buildkite.com/elastic/beats-xpack-metricbeat/builds/2698#018ef19e-b716-4727-b63d-641b7f0e1dfc (x-pack/metricbeat: Python (MODULE) Integration Tests ) it's a known failure see #39050 (comment))