Update Golang Cloud Functions in Functionbeat to use Golang 1.16.x on GCP

[kvch]

What does this PR do?

This PR adopts Golang 1.16 in cloud functions and removes vendoring in favour of go.mod files.

Why is it important?

Previously, GCP functions were broken.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have made corresponding change to the default configuration files
  - [ ] I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Closes #28132

[mergify]

This pull request does not have a backport label. Could you fix it @kvch? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

Pinging @elastic/agent (Team:Elastic-Agent)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-10-06T07:14:01.403+0000

• Duration: 60 min 39 sec

• Commit: 45e4206

Test stats 🧪

Test	Results
Failed	0
Passed	1413
Skipped	18
Total	1431

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

[endorama]

I'm trying to test this, but I keep getting a build failed error:

./functionbeat deploy -c functionbeat-cloud-sandbox.yml -e test
2021-10-06T16:01:04.487+0200	INFO	instance/beat.go:654	Home path: [/home/endorama/code/github.com/kvch/beats/x-pack/functionbeat] Config path: [/home/endorama/code/github.com/kvch/beats/x-pack/functionbeat] Data path: [/tmp] Logs path: [/tmp/logs]
2021-10-06T16:01:04.487+0200	INFO	instance/beat.go:662	Beat ID: 7f153246-eb40-4a08-94b8-e0de5d713d64
2021-10-06T16:01:07.492+0200	INFO	[add_cloud_metadata]	add_cloud_metadata/add_cloud_metadata.go:101	add_cloud_metadata: hosting provider type not detected.
Function: test, could not deploy, error: error while creating function (code: 3):
Build failed: go: github.com/elastic/beats/x-pack/functionbeat/provider/gcp/[email protected] requires
	github.com/elastic/beats/[email protected] requires
	github.com/Shopify/[email protected]: invalid version: unknown revision 000000000000; Error ID: 46183867
Fail to deploy 1 function(s)

What I did so far:

1. created a Pub/Sub topic to trigger the function
2. local build with mage build
3. package with mage buildPkgForFunctions
4. deploy to GCP with ./functionbeat deploy -c functionbeat-cloud-sandbox.yml -e test

But I get the error shown above. Any clue? Am I doing something wrong?

[kvch]

Closing as even if we release this, it will be supported only for a few weeks.

[endorama]

For reference, the alternative is to use GCP Dataflow (available since 7.15).

[chipzzz]

What about consumers that want to push to logstash :(