Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Original exporter IP in aggregated flow records is not Node IP #2336

Closed
srikartati opened this issue Jul 1, 2021 · 2 comments · Fixed by #2361
Closed

Original exporter IP in aggregated flow records is not Node IP #2336

srikartati opened this issue Jul 1, 2021 · 2 comments · Fixed by #2361
Labels
area/flow-visibility/aggregator Issues or PRs related to Flow Aggregator kind/bug Categorizes issue or PR as related to a bug.
Milestone
Antrea v1.2 release

Comments

@srikartati
Copy link
Member

srikartati commented Jul 1, 2021 •
edited
Loading

Describe the bug
Original exporter IP is not Node IP, but it is always the gateway of the Pod network on the Node in the Encap mode. This is expected as the Flow Exporter sends the flow records from the Antrea agent, which is on the host network to the Flow Aggregator which is on the Pod network.
It is not useful to have gateway IP in the aggregated flow records.

To Reproduce
Any aggregated flow record from Flow Aggregator on Antrea in Encap mode.

Expected
Having gateway IP as the original exporter IP is not very useful. We should think about the use of adding this field in Flow Aggregator.
As we already send the source Node name and destination Node name, which are the original exporters of the aggregated flow record. One option is to remove this as the original exporter IP does not make sense when correlating the flow records. The second option is to add the exporter IP in the flow record sent by the Flow Exporter and have two different fields corresponding to each exporter (source and destination).
I prefer option 1.

We have a similar field OriginalObservationDomainID. This also does not make much sense when we correlate the flow records exported from two nodes. This should also be removed if we go with the Option 1.

Actual behavior
Sample flow record:

egressNetworkPolicyRuleAction: 0 
tcpState: FIN_WAIT_2 
flowType: 3 
destinationClusterIPv4: 0.0.0.0 
**originalExporterIPv4Address: 10.10.1.1** 
originalObservationDomainId: 273090675 
octetDeltaCountFromSourceNode: 0 
octetTotalCountFromSourceNode: 0 
packetDeltaCountFromSourceNode: 0 
packetTotalCountFromSourceNode: 0

Versions:
Antrea main branch ToT

Additional context
If we go with the option 1 of removing the fields, then we can remove the additional templates that were recently added.
/cc @zyiou
@srikartati srikartati added kind/bug Categorizes issue or PR as related to a bug. area/flow-visibility/aggregator Issues or PRs related to Flow Aggregator labels Jul 1, 2021
@zyiou
Copy link
Contributor

zyiou commented Jul 1, 2021

Also prefer option 1 to remove both originalExporterIPv4Address/originalExporterIPv6Address and OriginalObservationDomainID since the information these fields reveal is either generated or can be shown from node name.

@antoninbas antoninbas added this to the Antrea v1.2 release milestone Jul 2, 2021
@antoninbas
Copy link
Contributor

Proposed solution sounds good to me

@srikartati srikartati mentioned this issue Jul 2, 2021
Merged
srikartati added a commit to srikartati/go-ipfix that referenced this issue Jul 2, 2021
@srikartati
Clear original export fields in the aggregation process
68560fc 
Original export fields should be removed in the intermediate process
in their current state.
In cases, where correlated flow records are there, current original export
fields do not make sense as records can come from two different
exporters.

More details in an Antrea issue, who is main user of go-ipfix library:
antrea-io/antrea#2336
srikartati added a commit to vmware/go-ipfix that referenced this issue Jul 2, 2021
@srikartati
Clear original export fields in the aggregation process
1a45478 
Original export fields should be removed in the intermediate process
in their current state.
In cases, where correlated flow records are there, current original export
fields do not make sense as records can come from two different
exporters.

More details in an Antrea issue, who is main user of go-ipfix library:
antrea-io/antrea#2336
zyiou pushed a commit to zyiou/go-ipfix that referenced this issue Jul 6, 2021
@srikartati @zyiou
Clear original export fields in the aggregation process
8ad90fa 
Original export fields should be removed in the intermediate process
in their current state.
In cases, where correlated flow records are there, current original export
fields do not make sense as records can come from two different
exporters.

More details in an Antrea issue, who is main user of go-ipfix library:
antrea-io/antrea#2336
zyiou pushed a commit to vmware/go-ipfix that referenced this issue Jul 7, 2021
@srikartati @zyiou
Clear original export fields in the aggregation process
8976af5 
Original export fields should be removed in the intermediate process
in their current state.
In cases, where correlated flow records are there, current original export
fields do not make sense as records can come from two different
exporters.

More details in an Antrea issue, who is main user of go-ipfix library:
antrea-io/antrea#2336
zyiou added a commit to zyiou/antrea that referenced this issue Jul 7, 2021
@zyiou
remove original exporter ip and obs domain ID from flow aggregator
fa27670 
This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.
fixes antrea-io#2336
@zyiou zyiou mentioned this issue Jul 7, 2021
Merged
zyiou added a commit to zyiou/antrea that referenced this issue Jul 7, 2021
@zyiou
remove original exporter ip and obs domain ID from flow aggregator
b34f3de 
This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.
fixes antrea-io#2336

Signed-off-by: zyiou <[email protected]>
zyiou added a commit to zyiou/antrea that referenced this issue Jul 7, 2021
@zyiou
remove original exporter ip and obs domain ID from flow aggregator
a4227aa 
This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.
fixes antrea-io#2336

Signed-off-by: zyiou <[email protected]>
zyiou added a commit to zyiou/antrea that referenced this issue Jul 8, 2021
@zyiou
remove original exporter ip and obs domain ID from flow aggregator
87c3545 
This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.
fixes antrea-io#2336

Signed-off-by: zyiou <[email protected]>
zyiou added a commit to zyiou/antrea that referenced this issue Jul 8, 2021
@zyiou
remove original exporter ip and obs domain ID from flow aggregator
e905056 
This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.
fixes antrea-io#2336

Signed-off-by: zyiou <[email protected]>
zyiou added a commit to zyiou/antrea that referenced this issue Jul 8, 2021
@zyiou
remove original exporter ip and obs domain ID from flow aggregator
8ac82ee 
This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.
fixes antrea-io#2336

Signed-off-by: zyiou <[email protected]>
antoninbas pushed a commit that referenced this issue Jul 9, 2021
@zyiou
Remove original exporter ip and obs domain ID from flow aggregator (#…
4895f6a 
…2361)

This commit removes originalExporterIPv4Address,
originalExporterIPv6Address and originalObservationDomainId from
flow aggregator and related templates for these fields.
It also bumps go-ipfix to v0.5.4.

Fixes #2336

Signed-off-by: zyiou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/flow-visibility/aggregator Issues or PRs related to Flow Aggregator kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
Antrea v1.2 release
Development

Successfully merging a pull request may close this issue.

[Flow Aggregator] Remove originalExporterAddress and originalObservationDomainId zyiou/antrea
3 participants
@antoninbas @srikartati @zyiou