Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] Additional utfbom fixes #25624

Closed
wants to merge 2 commits into from
Closed

[Filebeat] Additional utfbom fixes #25624

wants to merge 2 commits into from

Conversation

acamro
Copy link

@acamro acamro commented May 9, 2021 •
edited
Loading

Please label this PR with one of the following labels, depending on the scope:

  • Bug
  • Enhancement

What does this PR do?

Removes additional UTF marks in logs files:

  • UTF16BE
  • UTF16LE

Why is it important?

This fix several errors with some microsoft log files (ex. MS SQL Server 2019).

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Relates #1349

acamro added 2 commits May 8, 2021 22:00
@acamro
Removes additional UTF marks
69c7b2d 
Removes additional UTF (UTF16BE, UTF16LE) marks that cause errors with some microsoft log files.
@acamro
Test cases to check additional UTF marks
bf25bab 
Test cases to check additional UTF marks (UTF16BE, UTF16LE).
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 9, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 9, 2021 •
edited by jenkins-beats-ci bot
Loading

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: jsoriano commented: /test

  • Start Time: 2021-05-11T12:01:29.149+0000

  • Duration: 61 min 10 sec

  • Commit: bf25bab

Test stats 🧪

Test Results
Failed 0
Passed 46590
Skipped 4983
Total 51573

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 13

Expand to view the steps failures

Show only the first 10 steps failures

heartbeat-packaging-arm-arm - mage package
  • Took 7 min 0 sec . View more details on here
  • Description: mage package
journalbeat-packaging-arm-arm - mage package
  • Took 12 min 34 sec . View more details on here
  • Description: mage package
metricbeat-packaging-arm-arm - mage package
  • Took 12 min 18 sec . View more details on here
  • Description: mage package
packetbeat-packaging-arm-arm - mage package
  • Took 8 min 34 sec . View more details on here
  • Description: mage package
x-pack/auditbeat-packaging-arm-arm - mage package
  • Took 6 min 51 sec . View more details on here
  • Description: mage package
x-pack/elastic-agent-packaging-arm-arm - mage package
  • Took 20 min 17 sec . View more details on here
  • Description: mage package
x-pack/filebeat-packaging-arm-arm - mage package
  • Took 6 min 29 sec . View more details on here
  • Description: mage package
x-pack/heartbeat-packaging-arm-arm - mage package
  • Took 9 min 0 sec . View more details on here
  • Description: mage package
x-pack/metricbeat-packaging-arm-arm - mage package
  • Took 7 min 5 sec . View more details on here
  • Description: mage package
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output 

[2021-05-11T13:00:50.763Z]  OS/Arch:           linux/amd64
[2021-05-11T13:00:50.763Z]  Context:           default
[2021-05-11T13:00:50.763Z]  Experimental:      true
[2021-05-11T13:00:50.763Z] 
[2021-05-11T13:00:50.763Z] Server: Docker Engine - Community
[2021-05-11T13:00:50.763Z]  Engine:
[2021-05-11T13:00:50.763Z]   Version:          20.10.3
[2021-05-11T13:00:50.763Z]   API version:      1.41 (minimum version 1.12)
[2021-05-11T13:00:50.763Z]   Go version:       go1.13.15
[2021-05-11T13:00:50.763Z]   Git commit:       46229ca
[2021-05-11T13:00:50.764Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-05-11T13:00:50.765Z]   OS/Arch:          linux/amd64
[2021-05-11T13:00:50.765Z]   Experimental:     false
[2021-05-11T13:00:50.765Z]  containerd:
[2021-05-11T13:00:50.765Z]   Version:          1.4.4
[2021-05-11T13:00:50.765Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-05-11T13:00:50.765Z]  runc:
[2021-05-11T13:00:50.765Z]   Version:          1.0.0-rc93
[2021-05-11T13:00:50.765Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-05-11T13:00:50.765Z]  docker-init:
[2021-05-11T13:00:50.765Z]   Version:          0.19.0
[2021-05-11T13:00:50.765Z]   GitCommit:        de40ad0
[2021-05-11T13:00:50.765Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-05-11T13:00:50.765Z] Unable to find image 'alpine:3.4' locally
[2021-05-11T13:00:51.337Z] 3.4: Pulling from library/alpine
[2021-05-11T13:00:51.599Z] c1e54eec4b57: Pulling fs layer
[2021-05-11T13:00:51.862Z] c1e54eec4b57: Download complete
[2021-05-11T13:00:52.123Z] c1e54eec4b57: Pull complete
[2021-05-11T13:00:52.123Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-05-11T13:00:52.123Z] Status: Downloaded newer image for alpine:3.4
[2021-05-11T13:00:54.044Z] Change permissions with write access of all files inside the specific folder
[2021-05-11T13:00:55.474Z] Running in /var/lib/jenkins/workspace/PR-25624-2-0a5090a7-be7d-45b1-8a00-e72bf67c64e0/src/github.com/elastic/beats/build
[2021-05-11T13:00:55.788Z] + rm -rf ve
[2021-05-11T13:00:55.788Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-05-11T13:00:56.129Z] + python .ci/scripts/pre_archive_test.py
[2021-05-11T13:01:11.061Z] Copy ./x-pack/metricbeat/build into build/x-pack/metricbeat/build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/metricbeat/build/golang-crossbuild into build/x-pack/metricbeat/build/golang-crossbuild
[2021-05-11T13:01:11.061Z] Copy ./x-pack/metricbeat/build/package/metricbeat/metricbeat-linux-amd64.docker/docker-build into build/x-pack/metricbeat/build/package/metricbeat/metricbeat-linux-amd64.docker/docker-build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/metricbeat/build/package/metricbeat-ubi8/metricbeat-linux-amd64.docker/docker-build into build/x-pack/metricbeat/build/package/metricbeat-ubi8/metricbeat-linux-amd64.docker/docker-build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/heartbeat/build into build/x-pack/heartbeat/build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/heartbeat/build/golang-crossbuild into build/x-pack/heartbeat/build/golang-crossbuild
[2021-05-11T13:01:11.061Z] Copy ./x-pack/elastic-agent/build into build/x-pack/elastic-agent/build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/elastic-agent/build/golang-crossbuild into build/x-pack/elastic-agent/build/golang-crossbuild
[2021-05-11T13:01:11.061Z] Copy ./x-pack/elastic-agent/build/package/elastic-agent-ubi8/elastic-agent-linux-amd64.docker/docker-build into build/x-pack/elastic-agent/build/package/elastic-agent-ubi8/elastic-agent-linux-amd64.docker/docker-build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/elastic-agent/build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build into build/x-pack/elastic-agent/build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-05-11T13:01:11.061Z] Copy ./x-pack/filebeat/build/golang-crossbuild into build/x-pack/filebeat/build/golang-crossbuild
[2021-05-11T13:01:11.061Z] Copy ./x-pack/filebeat/build/package/filebeat-ubi8/filebeat-linux-amd64.docker/docker-build into build/x-pack/filebeat/build/package/filebeat-ubi8/filebeat-linux-amd64.docker/docker-build
[2021-05-11T13:01:11.062Z] Copy ./x-pack/filebeat/build/package/filebeat/filebeat-linux-amd64.docker/docker-build into build/x-pack/filebeat/build/package/filebeat/filebeat-linux-amd64.docker/docker-build
[2021-05-11T13:01:11.085Z] Running in /var/lib/jenkins/workspace/PR-25624-2-0a5090a7-be7d-45b1-8a00-e72bf67c64e0/src/github.com/elastic/beats/build
[2021-05-11T13:01:11.115Z] Recording test results
[2021-05-11T13:01:15.151Z] None of the test reports contained any result
[2021-05-11T13:01:15.160Z] [Checks API] No suitable checks publisher found.
[2021-05-11T13:01:15.528Z] + go clean -modcache
[2021-05-11T13:01:19.170Z] Cleaning up /var/lib/jenkins/workspace/PR-25624-2-0a5090a7-be7d-45b1-8a00-e72bf67c64e0
[2021-05-11T13:01:19.170Z] Client: Docker Engine - Community
[2021-05-11T13:01:19.170Z]  Version:           20.10.3
[2021-05-11T13:01:19.170Z]  API version:       1.41
[2021-05-11T13:01:19.170Z]  Go version:        go1.13.15
[2021-05-11T13:01:19.170Z]  Git commit:        48d30b5
[2021-05-11T13:01:19.170Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-05-11T13:01:19.170Z]  OS/Arch:           linux/amd64
[2021-05-11T13:01:19.170Z]  Context:           default
[2021-05-11T13:01:19.170Z]  Experimental:      true
[2021-05-11T13:01:19.170Z] 
[2021-05-11T13:01:19.170Z] Server: Docker Engine - Community
[2021-05-11T13:01:19.170Z]  Engine:
[2021-05-11T13:01:19.170Z]   Version:          20.10.3
[2021-05-11T13:01:19.170Z]   API version:      1.41 (minimum version 1.12)
[2021-05-11T13:01:19.170Z]   Go version:       go1.13.15
[2021-05-11T13:01:19.170Z]   Git commit:       46229ca
[2021-05-11T13:01:19.170Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-05-11T13:01:19.170Z]   OS/Arch:          linux/amd64
[2021-05-11T13:01:19.170Z]   Experimental:     false
[2021-05-11T13:01:19.170Z]  containerd:
[2021-05-11T13:01:19.170Z]   Version:          1.4.4
[2021-05-11T13:01:19.170Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-05-11T13:01:19.170Z]  runc:
[2021-05-11T13:01:19.170Z]   Version:          1.0.0-rc93
[2021-05-11T13:01:19.170Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-05-11T13:01:19.170Z]  docker-init:
[2021-05-11T13:01:19.170Z]   Version:          0.19.0
[2021-05-11T13:01:19.170Z]   GitCommit:        de40ad0
[2021-05-11T13:01:19.170Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-05-11T13:02:26.932Z] Change permissions with write access of all files inside the specific folder
[2021-05-11T13:02:26.966Z] Running in /var/lib/jenkins/workspace/PR-25624-2-0a5090a7-be7d-45b1-8a00-e72bf67c64e0
[2021-05-11T13:02:33.209Z] + gsutil --version
[2021-05-11T13:02:34.640Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-05-11T13:02:34.972Z] + gcloud auth activate-service-account --key-file ****
[2021-05-11T13:02:35.543Z] Activated service account credentials for: [[email protected]]
[2021-05-11T13:02:35.879Z] + gsutil -m -q cp -a public-read eC1wYWNrL2VsYXN0aWMtYWdlbnQtcGFja2FnaW5nLWxpbnV4YmYyNWJhYjllM2VlNjg1ZGI1NjMwOWEzOWM3NWY3ZWQ3ZDdlNzg1OA gs://beats-ci-temp/ci/cache/
[2021-05-11T13:02:37.516Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-05-11T13:02:37.670Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-25624/src/github.com/elastic/beats
[2021-05-11T13:02:38.149Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-25624
[2021-05-11T13:02:38.202Z] [INFO] getVaultSecret: Getting secrets
[2021-05-11T13:02:38.251Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-05-11T13:02:39.032Z] + chmod 755 generate-build-data.sh
[2021-05-11T13:02:39.032Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25624/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25624/runs/2 FAILURE 3669616
[2021-05-11T13:02:39.032Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25624/runs/2/steps/?limit=10000 -o steps-info.json
[2021-05-11T13:02:53.794Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25624/runs/2/tests/?status=FAILED -o tests-errors.json

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 46590
Skipped 4983
Total 51573

@acamro acamro changed the title Additional utfbom fixes [Filebeat] Additional utfbom fixes May 9, 2021
@jsoriano jsoriano added the Team:Elastic-Agent Label for the Agent team label May 11, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/agent (Team:Agent)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 11, 2021
@jsoriano
Copy link
Member

/test

@noobwu
Copy link

noobwu commented Nov 7, 2021

file custom module Additional utfbom

@botelastic
Copy link

botelastic bot commented Jan 31, 2022

Hi!
We just realized that we haven't looked into this PR in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it in as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jan 31, 2022
@botelastic
Copy link

botelastic bot commented Mar 2, 2022

Hi!
This PR has been stale for a while and we're going to close it as part of our cleanup procedure.
We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team.
Feel free to re-open this PR if you think it should stay open and is worth rebasing.
Thank you for your contribution!

@botelastic botelastic bot closed this Mar 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Stalled Team:Elastic-Agent Label for the Agent team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@acamro @elasticmachine @jsoriano @noobwu