From be9c33385dbb9bc080d7fe5ceeb854acb1af85ed Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 15 Feb 2023 21:44:46 -0800 Subject: [PATCH 1/8] [Sonic] PL on NSG and Inbound flows --- documentation/general/dash-sonic-hld.md | 92 +++++++++++++++++++++++-- 1 file changed, 86 insertions(+), 6 deletions(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index 4e71430eb..462431c5c 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -200,6 +200,8 @@ It is worth noting that CA-PA mapping table shall be used for both encap and dec ST/PL is employed for scenarios like multiple different customers want to access a common shared resource (e.g storage). This shall not fall into the regular Vnet packet path or Vnet peering path and hence a Private Endpoint is assigned for such accesses, as part of ENI routing or VNET's mapping tables. The lookup happens as described in the above sections, but actions are different. For ST/PL, actions include IPv4 to IPv6 transpositions and special routing/mapping lookups for encapsulation. By having packet transpositions, Service Tunnel feature provides the capability of encoding “region id”, “vnet id”, “subnet id” etc via packet transformation. IPv6 transformation includes last 32 bits of the IPv6 packet as IPv4 address, while the remaining 96 bits of the IPv6 packet is used for encoding. Private Link feature is an extension to Service Tunnel feature and enables customers to access public facing shared services via their private IP addresses within their vnet. More details on traffic flow is captured in the example section. **ST/PL Inbound flow**: Using the outbound unified flow, the reverse transposition (inbound unified flow) is created. If no inbound flow is created, the packet shall be dropped if it does not match any existing inbound routing rule. There is no inbound policy based lookup expected for ST/PL scenarios. When FastPath kicks in, the respective outbound and inbound unified flows shall be modified accordingly. + +ST encap shall be either NVGRE or Vxlan. The same encapsulation is expected on the inbound flow. # 3 Modules Design @@ -351,6 +353,24 @@ encap_type = encap type depends on the action_type - {vxlan, nvgre vni = vni value associated with the corresponding action. Applicable if encap_type is specified. ``` +### 3.2.5.1 ROUTING APPLIANCE + +``` +DASH_ROUTING_APPLIANCE_TABLE:{{appliance_id}}: + "appliance_guid":{{string}} + "addresses": {{list of addresses}} + "encap_type": {{encap type}} + "vni": {{vni}} +``` + +``` +key = DASH_ROUTING_APPLIANCE_TABLE:appliance_id; Used for PL NSG +; field = value +addresses = list of addresses used for ECMP across appliances +encap_type = encap type depends on the action_type - {vxlan, nvgre} +vni = vni value associated with the corresponding action. +``` + ### 3.2.6 APPLIANCE ``` @@ -658,16 +678,19 @@ Underlay attributes on a DASH appliance shall be programmed similar to Sonic swi Note that *only* default route is expected from the peer BGP and appliance is _not_ expected to allocate an LPM resource for underlay. Implementation can choose whether to forward the packet on the same port it is received or do forwarding based on route and next-hop entry. Same is applicable for ECMP where the implementation can perform 5-tuple hashing or forward the "return" traffic on the same port it has received the original packet. -### 3.3.6 Memory footprints +### 3.3.6 Encap behavior +Default DSCP and TTL behaviour for Vxlan encap shall be "pipe" model (similar to SAI_TUNNEL_DSCP_MODE_PIPE_MODEL, SAI_TUNNEL_TTL_MODE_PIPE_MODEL). Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. For routing action "direct", for e.g internet traffic, the DSCP value shall be reset to 0 even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with value 0. Outer TTL value shall be default set to 64. + +### 3.3.7 Memory footprints -#### 3.3.6.1 SONiC memory usage +#### 3.3.7.1 SONiC memory usage | Running components | Memory usage | |--|--| |Base Debian OS | 159MB | |Base Debian OS + docker containers | 1.3GB | -#### 3.3.6.2 SONiC docker containers memory usage +#### 3.3.7.2 SONiC docker containers memory usage |Container| Memory usage | |--|--| @@ -915,6 +938,14 @@ For the example configuration above, the following is a brief explanation of loo d. Packet gets transformed as: Overlay SIP fd00:108:0:d204:0:200::0a01:101, Overlay DIP 2603:10e1:100:2::3c01:201 e. Second Action is Static NVGRE encap. f. Since underlay sip/dip is specified in the LPM table, It shall use Dst IP (25.1.2.1), Src IP (30.1.2.1) + g. Thus the packet is send with two headers - One Inner IPv6 and another Outer IPv4 + h. Inbound packet for this flow: + h.1 The return packet for the above flow shall land on an SLB MUX (30.1.2.1) as NVGRE packet + h.2 SLB MUX (30.1.2.1) shall encapsulate this to another header with standard GRE key, say 100 and forwards to Appliance VIP + h.3 Appliance shall first decapsulate the outer header and map it to a flow + h.4 Second header's dst mac shall correspond to ENI MAC, as overwritten by SLB MUX + h.5 Third header shall be the transpositioned IPv6 header + i. Note: This flow fixup shall be done when Fastpath kicks in with ICMP Redirect. 3. Packet destined to 70.1.2.1 from 10.1.1.1: a. LPM lookup hits for entry 70.1.2.0/24 @@ -940,8 +971,33 @@ For the example configuration above, the following is a brief explanation of loo "encap_type": "nvgre" "key":"100" } ], + "OP": "SET", + "DASH_ROUTING_TYPE_TABLE:privatelinknsg": [ + { + "name": "action1", + "action_type": "4to6", + }, + { + "name": "action2", + "action_type": "staticencap", + "encap_type": "nvgre" + "key":"100" + }, + { + "name": "action3", + "action_type": "appliance", + } ], "OP": "SET" }, + { + DASH_ROUTING_APPLIANCE_TABLE:22: { + "appliance_guid":"497f23d7-f0ac-4c99", + "addresses": "100.8.1.2", + "encap_type": "vxlan", + "vni": 101 + }, + "OP": "SET" + }, { "DASH_ENI_TABLE:F4939FEFC47E": { "eni_id": "497f23d7-f0ac-4c99-a98f-59b470e8c7bd", @@ -972,7 +1028,7 @@ For the example configuration above, the following is a brief explanation of loo "OP": "SET" }, { - "DASH_ROUTE_TABLE:F4939FEFC47E:10.2.0.6/32": { + "DASH_ROUTE_TABLE:F4939FEFC47E:10.2.0.6/24": { "action_type":"vnet", "vnet":"Vnet1" }, @@ -987,6 +1043,17 @@ For the example configuration above, the following is a brief explanation of loo "overlay_dip":"2603:10e1:100:2::3402:206", }, "OP": "SET" + }, + { + "DASH_VNET_MAPPING_TABLE:Vnet1:10.2.0.9": { + "routing_type":"privatelinknsg", + "mac_address":"F9-22-83-99-22-A2", + "underlay_ip":"50.2.2.6", + "overlay_sip":"fd40:108:0:d204:0:200::0", + "overlay_dip":"2603:10e1:100:2::3402:206", + "routing_appliance_id":22 + }, + "OP": "SET" } ] ``` @@ -1008,7 +1075,7 @@ For the example configuration above, the following is a brief explanation of loo g. Underlay DIP shall be 50.1.2.3 (from mapping), Underlay SIP shall be 55.1.2.3 (from ENI) 2. Packet destined to 10.2.0.6 from 10.1.1.2: - a. LPM lookup hits for entry 10.2.0.6/32 + a. LPM lookup hits for entry 10.2.0.6/24 b. The action in this case is "vnet" c. Next lookup is in the mapping table and mapping table action here is "privatelink" d. First Action for "privatelink" is 4to6 transposition @@ -1017,4 +1084,17 @@ For the example configuration above, the following is a brief explanation of loo Overlay DIP 2603:10e1:100:2::3402:206 (No transformation, provided as part of mapping) f. Second Action is Static NVGRE encap with GRE key '100'. g. Underlay DIP shall be 50.2.2.6 (from mapping), Underlay SIP shall be 55.1.2.3 (from ENI) - + + 3. Packet destined to 10.2.0.9 from 10.1.1.2: + a. LPM lookup hits for entry 10.2.0.6/24 + b. The action in this case is "vnet" + c. Next lookup is in the mapping table and mapping table action here is "privatelinknsg" + d. First Action for "privatelink" is 4to6 transposition + e. Packet gets transformed as: + For Overlay SIP, using ENI's "pl_sip_encoding": "0x0020000000000a0b0c0d0a0b/0x002000000000ffffffffffff" -> Overlay SIP fd30:108:0:0a0b:0c0d:0a0b:a01:102; + Overlay DIP 2603:10e1:100:2::3402:206 (No transformation, provided as part of mapping) + f. Second Action is Static NVGRE encap with GRE key '100'. + g. Underlay DIP shall be 50.2.2.6 (from mapping), Underlay SIP shall be 55.1.2.3 (from ENI) + h. Third Action is Appliance Encap for id 22 + i. Packet shall be encapsulated with Outer DIP as 100.8.1.2 and SIP as VIP of this originating appliance card with VNI of 101. + j. Inbound flow shall be similar to PL and outer encap shall be of the SLB MUX and not of the NSG appliance. From 7a64ef955ee9126bc268989009bff6f7380b2dcd Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 15 Feb 2023 22:00:53 -0800 Subject: [PATCH 2/8] Update dash-sonic-hld.md --- documentation/general/dash-sonic-hld.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index 462431c5c..f09279407 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -201,7 +201,7 @@ It is worth noting that CA-PA mapping table shall be used for both encap and dec ST/PL is employed for scenarios like multiple different customers want to access a common shared resource (e.g storage). This shall not fall into the regular Vnet packet path or Vnet peering path and hence a Private Endpoint is assigned for such accesses, as part of ENI routing or VNET's mapping tables. The lookup happens as described in the above sections, but actions are different. For ST/PL, actions include IPv4 to IPv6 transpositions and special routing/mapping lookups for encapsulation. By having packet transpositions, Service Tunnel feature provides the capability of encoding “region id”, “vnet id”, “subnet id” etc via packet transformation. IPv6 transformation includes last 32 bits of the IPv6 packet as IPv4 address, while the remaining 96 bits of the IPv6 packet is used for encoding. Private Link feature is an extension to Service Tunnel feature and enables customers to access public facing shared services via their private IP addresses within their vnet. More details on traffic flow is captured in the example section. **ST/PL Inbound flow**: Using the outbound unified flow, the reverse transposition (inbound unified flow) is created. If no inbound flow is created, the packet shall be dropped if it does not match any existing inbound routing rule. There is no inbound policy based lookup expected for ST/PL scenarios. When FastPath kicks in, the respective outbound and inbound unified flows shall be modified accordingly. -ST encap shall be either NVGRE or Vxlan. The same encapsulation is expected on the inbound flow. +ST encap shall be either nvgre or vxlan. The same encapsulation is expected on the inbound flow. # 3 Modules Design @@ -679,7 +679,7 @@ Underlay attributes on a DASH appliance shall be programmed similar to Sonic swi Note that *only* default route is expected from the peer BGP and appliance is _not_ expected to allocate an LPM resource for underlay. Implementation can choose whether to forward the packet on the same port it is received or do forwarding based on route and next-hop entry. Same is applicable for ECMP where the implementation can perform 5-tuple hashing or forward the "return" traffic on the same port it has received the original packet. ### 3.3.6 Encap behavior -Default DSCP and TTL behaviour for Vxlan encap shall be "pipe" model (similar to SAI_TUNNEL_DSCP_MODE_PIPE_MODEL, SAI_TUNNEL_TTL_MODE_PIPE_MODEL). Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. For routing action "direct", for e.g internet traffic, the DSCP value shall be reset to 0 even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with value 0. Outer TTL value shall be default set to 64. +Default DSCP and TTL behavior for vxlan or nvgre encap shall be "pipe" model (similar to SAI_TUNNEL_DSCP_MODE_PIPE_MODEL, SAI_TUNNEL_TTL_MODE_PIPE_MODEL). Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to 0 even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with value 0. Outer TTL value shall be default set to 64. ### 3.3.7 Memory footprints From b37bc37febb4c94962d96130444f1ace031fc392 Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Thu, 16 Feb 2023 08:53:55 -0800 Subject: [PATCH 3/8] Update .wordlist.txt --- .wordlist.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.wordlist.txt b/.wordlist.txt index 006b579dd..8a930d6c7 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -356,6 +356,8 @@ num NumberOfFlowResimulated NVA NVidia +nvgre +Nvgre NVMe observability OCP From 54f55eb06f3cf6b45d5f3a9253926e9c08c8bd25 Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 22 Feb 2023 15:06:37 -0800 Subject: [PATCH 4/8] Clarify the encap behavior based on community feedback --- documentation/general/dash-sonic-hld.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index f09279407..d6fcf912d 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -679,7 +679,7 @@ Underlay attributes on a DASH appliance shall be programmed similar to Sonic swi Note that *only* default route is expected from the peer BGP and appliance is _not_ expected to allocate an LPM resource for underlay. Implementation can choose whether to forward the packet on the same port it is received or do forwarding based on route and next-hop entry. Same is applicable for ECMP where the implementation can perform 5-tuple hashing or forward the "return" traffic on the same port it has received the original packet. ### 3.3.6 Encap behavior -Default DSCP and TTL behavior for vxlan or nvgre encap shall be "pipe" model (similar to SAI_TUNNEL_DSCP_MODE_PIPE_MODEL, SAI_TUNNEL_TTL_MODE_PIPE_MODEL). Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to 0 even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with value 0. Outer TTL value shall be default set to 64. +Default DSCP behavior for vxlan or nvgre encap shall be "uniform" model (similar to SAI_TUNNEL_DSCP_MODE_UNIFORM_MODEL) and TTL behavior for encap shall be "pipe" model (similar to SAI_TUNNEL_TTL_MODE_PIPE_MODEL). However for DSCP, pipleline must copy the DSCP value to the new encap outer header from the original incoming packet *before* decap was done and not from the customer packet header. Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to original header's DSCP value even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with same value that is recieved in the original outer header. Outer TTL value shall be default set to 64. ### 3.3.7 Memory footprints From 9370fd2cdb87f8f901e783af5bfc06cc6ae95169 Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 22 Feb 2023 15:08:35 -0800 Subject: [PATCH 5/8] Fix spelling --- documentation/general/dash-sonic-hld.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index d6fcf912d..7e6a1ed82 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -679,7 +679,7 @@ Underlay attributes on a DASH appliance shall be programmed similar to Sonic swi Note that *only* default route is expected from the peer BGP and appliance is _not_ expected to allocate an LPM resource for underlay. Implementation can choose whether to forward the packet on the same port it is received or do forwarding based on route and next-hop entry. Same is applicable for ECMP where the implementation can perform 5-tuple hashing or forward the "return" traffic on the same port it has received the original packet. ### 3.3.6 Encap behavior -Default DSCP behavior for vxlan or nvgre encap shall be "uniform" model (similar to SAI_TUNNEL_DSCP_MODE_UNIFORM_MODEL) and TTL behavior for encap shall be "pipe" model (similar to SAI_TUNNEL_TTL_MODE_PIPE_MODEL). However for DSCP, pipleline must copy the DSCP value to the new encap outer header from the original incoming packet *before* decap was done and not from the customer packet header. Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to original header's DSCP value even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with same value that is recieved in the original outer header. Outer TTL value shall be default set to 64. +Default DSCP behavior for vxlan or nvgre encap shall be "uniform" model (similar to SAI_TUNNEL_DSCP_MODE_UNIFORM_MODEL) and TTL behavior for encap shall be "pipe" model (similar to SAI_TUNNEL_TTL_MODE_PIPE_MODEL). However for DSCP, pipleline must copy the DSCP value to the new encap outer header from the original incoming packet *before* decap was done and not from the customer packet header. Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to original header's DSCP value even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with same value that is received in the original outer header. Outer TTL value shall be default set to 64. ### 3.3.7 Memory footprints From d6b086fd16e0b70db4b42bc4b2eeec0ed1aebc7b Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 22 Feb 2023 15:20:40 -0800 Subject: [PATCH 6/8] Update dash-sonic-hld.md --- documentation/general/dash-sonic-hld.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index 7e6a1ed82..fd32ba176 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -679,7 +679,7 @@ Underlay attributes on a DASH appliance shall be programmed similar to Sonic swi Note that *only* default route is expected from the peer BGP and appliance is _not_ expected to allocate an LPM resource for underlay. Implementation can choose whether to forward the packet on the same port it is received or do forwarding based on route and next-hop entry. Same is applicable for ECMP where the implementation can perform 5-tuple hashing or forward the "return" traffic on the same port it has received the original packet. ### 3.3.6 Encap behavior -Default DSCP behavior for vxlan or nvgre encap shall be "uniform" model (similar to SAI_TUNNEL_DSCP_MODE_UNIFORM_MODEL) and TTL behavior for encap shall be "pipe" model (similar to SAI_TUNNEL_TTL_MODE_PIPE_MODEL). However for DSCP, pipleline must copy the DSCP value to the new encap outer header from the original incoming packet *before* decap was done and not from the customer packet header. Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to original header's DSCP value even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with same value that is received in the original outer header. Outer TTL value shall be default set to 64. +Default DSCP behavior for vxlan or nvgre encap shall be "uniform" model (similar to SAI_TUNNEL_DSCP_MODE_UNIFORM_MODEL) and TTL behavior for encap shall be "pipe" model (similar to SAI_TUNNEL_TTL_MODE_PIPE_MODEL). However for DSCP, pipeline must copy the DSCP value to the new encap outer header from the original incoming packet *before* decap was done and not from the customer packet header. Appliance shall not modify the DSCP or TTL values in the inner packet (customer packet) during an encap and shall ensure outer header values are independent of the customer generated headers. Similarly, for routing action type "direct", for e.g internet traffic, the DSCP value shall be reset to original header's DSCP value even-if original packet arrives with a non-zero value effectively ensuring all outbound traffic from the appliance is transmitted with same value that is received in the original outer header. Outer TTL value shall be default set to 64. ### 3.3.7 Memory footprints From 32c17e76b99432f1c93972ef0a5253fe16dcbf64 Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 22 Mar 2023 09:43:19 -0700 Subject: [PATCH 7/8] Address review comments --- documentation/general/dash-sonic-hld.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index 306a343bf..78884d8ac 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -1124,14 +1124,15 @@ For the example configuration above, the following is a brief explanation of loo d. Packet gets transformed as: Overlay SIP fd00:108:0:d204:0:200::0a01:101, Overlay DIP 2603:10e1:100:2::3c01:201 e. Second Action is Static NVGRE encap. f. Since underlay sip/dip is specified in the LPM table, It shall use Dst IP (25.1.2.1), Src IP (30.1.2.1) - g. Thus the packet is send with two headers - One Inner IPv6 and another Outer IPv4 + g. Thus the packet is sent with two headers - One Inner IPv6 and another Outer IPv4 h. Inbound packet for this flow: - h.1 The return packet for the above flow shall land on an SLB MUX (30.1.2.1) as NVGRE packet - h.2 SLB MUX (30.1.2.1) shall encapsulate this to another header with standard GRE key, say 100 and forwards to Appliance VIP - h.3 Appliance shall first decapsulate the outer header and map it to a flow - h.4 Second header's dst mac shall correspond to ENI MAC, as overwritten by SLB MUX - h.5 Third header shall be the transpositioned IPv6 header - i. Note: This flow fixup shall be done when Fastpath kicks in with ICMP Redirect. + h.1 The return packet for the above flow shall land on an SLB MUX (30.1.2.1) as NVGRE packet + h.2 SLB MUX (30.1.2.1) shall encapsulate this to another header with standard GRE key, say 100 and forwards to Appliance PA + h.3 Appliance PA is the unique IP advertised by the Appliance card, say via BGP for reachability. + h.3 Appliance shall first decapsulate the outer header and map it to a flow + h.4 Second header's dst mac shall correspond to ENI MAC, as overwritten by SLB MUX + h.5 Third header shall be the transpositioned IPv6 header + i. Note: This flow fixup shall be done when Fastpath kicks in with ICMP Redirect, and packets ingress with two headers. 3. Packet destined to 70.1.2.1 from 10.1.1.1: a. LPM lookup hits for entry 70.1.2.0/24 @@ -1274,7 +1275,7 @@ For the example configuration above, the following is a brief explanation of loo For Overlay SIP, using ENI's "pl_sip_encoding": "0x0020000000000a0b0c0d0a0b/0x002000000000ffffffffffff" -> Overlay SIP fd30:108:0:0a0b:0c0d:0a0b:a01:101 using the following logic: 1. fv = (fd40:108:0:d204:0:200::0 & !0x002000000000ffffffffffff) (first 96 bits based on provided mask length) 2. result = fv | 0x0020000000000a0b0c0d0a0b (first 96 bits based on the provided mask length) - 3. result = result | ca (last 32 bits if its set to 0 in mapping, implicit conversion) + 3. result = result | source CA (last 32 bits if its set to 0 in mapping, implicit conversion) Overlay DIP 2603:10e1:100:2::3401:203 (No transformation, provided as part of mapping) f. Second Action is Static NVGRE encap with GRE key '100'. g. Underlay DIP shall be 50.1.2.3 (from mapping), Underlay SIP shall be 55.1.2.3 (from ENI) From 4acdc7a1f07509cdc2e29c7b191c33c65747a701 Mon Sep 17 00:00:00 2001 From: Prince Sunny Date: Wed, 22 Mar 2023 09:44:30 -0700 Subject: [PATCH 8/8] Update dash-sonic-hld.md --- documentation/general/dash-sonic-hld.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index 78884d8ac..860864cfc 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -1129,9 +1129,9 @@ For the example configuration above, the following is a brief explanation of loo h.1 The return packet for the above flow shall land on an SLB MUX (30.1.2.1) as NVGRE packet h.2 SLB MUX (30.1.2.1) shall encapsulate this to another header with standard GRE key, say 100 and forwards to Appliance PA h.3 Appliance PA is the unique IP advertised by the Appliance card, say via BGP for reachability. - h.3 Appliance shall first decapsulate the outer header and map it to a flow - h.4 Second header's dst mac shall correspond to ENI MAC, as overwritten by SLB MUX - h.5 Third header shall be the transpositioned IPv6 header + h.4 Appliance shall first decapsulate the outer header and map it to a flow + h.5 Second header's dst mac shall correspond to ENI MAC, as overwritten by SLB MUX + h.6 Third header shall be the transpositioned IPv6 header i. Note: This flow fixup shall be done when Fastpath kicks in with ICMP Redirect, and packets ingress with two headers. 3. Packet destined to 70.1.2.1 from 10.1.1.1: