Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the .address fields for ambiguous address prior to extracting IP & domain #10141

Merged
merged 7 commits into from
Jan 18, 2019

Conversation

webmat
Copy link
Contributor

@webmat webmat commented Jan 17, 2019

This migration has been done in a few modules already. This completes the transition for the modules transitioned prior to ECS 1.0.0 Beta 2.

Modules/filesets affected

  • apache.access (apache2.access.remote_ip)
  • haproxy.log (haproxy.client.ip)

Noticed

HAProxy was migrated prior to ECS Beta 2. A few more fields should be migrated, opened #10143 for this.

@webmat webmat requested review from a team as code owners January 17, 2019 03:06
@webmat webmat self-assigned this Jan 17, 2019
@webmat webmat requested a review from ruflin January 17, 2019 03:30
@ruflin ruflin mentioned this pull request Jan 17, 2019
@webmat
Copy link
Contributor Author

webmat commented Jan 17, 2019

Filebeat tests on both CI systems are hanging. I'll investigate this one some more.

@webmat webmat force-pushed the ecs-ambiguous-address branch 2 times, most recently from f2e5893 to 5ac0c1c Compare January 17, 2019 21:51
Mathieu Martin added 5 commits January 17, 2019 21:14
Note that I'm modifying the grok because the only thing expected in this field is either an IP or a socket path (e.g. for admin socket). And ECS doesn't define a place for socket paths in src/dst. IPs are never resolved to hostnames in HAProxy.
Can't actually get an error message for these timeouts, even when reproducing locally. Out of ideas.

This reverts commit a78ebeb419cfa162551891846d35ba22c0bf8757.
@webmat webmat force-pushed the ecs-ambiguous-address branch from e022bb6 to 3bd8fd5 Compare January 18, 2019 02:14
@ruflin
Copy link
Contributor

ruflin commented Jan 18, 2019

@webmat Please ignore the Jenkins CI for now.

Mathieu Martin added 2 commits January 18, 2019 12:30
- apache2.access.remote_ip points to source.address
- apache.access.remote_ip is simply removed, since it was never publicly released

My problems initially were caused by an alias pointing to an alias (apache2.access.remote_ip => apache.access.remote_ip => source.address)
@webmat
Copy link
Contributor Author

webmat commented Jan 18, 2019

Pretty sure I found my problem with CI, and it was legit.

I was simply aliasing apache.access.remote_ip to source.address. However we had the field names prior to the module rename still aliasing apache2.access.remote_ip to apache.access.remote_ip, which meant an alias was pointing to an alias.

The alias is now directly apache2.access.remote_ip => source.address, and apache.access.remote_ip has been deleted, since the module post-rename has never been released publicly anyway.

@webmat webmat merged commit 4a837b7 into elastic:master Jan 18, 2019
@webmat webmat deleted the ecs-ambiguous-address branch January 18, 2019 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants