[multi-asic][vs]: Add new multi-asic vs hwsku with four asics #6558

SuvarnaMeenakshi · 2021-01-25T20:08:56Z

- Why I did it

Current mutli-asic vs hwsku consists of 6 asics with each asic having 32 interfaces. When bringing this up, below issue was seen:
When all 32 interfaces(sonic interfaces and linux interface) are set to 9100 mtu, DMA error is seen "DMA: Out of SW-IOMMU space for 4096 bytes at device 0000:06:03.0" which can be fixed by updating swiotlb=65536 in /host/grub/grub.cfg .

In order to keep multi-asic VS lighter and easier to bring up and test, new hwsku 'msft_four_asic_vs' is added to represent 4-asic hwsku with 2 frontend asics and 2 backend asics and each asic having 8 interfaces interconnected by port-channels.

- How I did it

Add msft_four_asic_hwsku directory to have the right number of directories (4) and update port_config.ini and lanemap.ini files to include 8 ports information.
Add topology.sh script to create the internal asic-asic connectivity.

- How to verify it

Update asic.conf with the 4 asic information as below and build sonic-vs.img:
NUM_ASIC=4
DEV_ID_ASIC_0=0
DEV_ID_ASIC_1=1
DEV_ID_ASIC_2=2
DEV_ID_ASIC_3=3
Modify sonic_multiasic.xml to have 8 front panel interfaces.
create virtual switch using "sudo virsh sonic_mutliasic.xml" command.
Start topology service and Load config_db files for switch and each asic.
Ensure that that all internal interfaces and port_channels are coming up.
multi-asic vs testbed:
Bring up mutli-asic VS testbed with a multi-asic image(asic.conf updated to 4 asics) and using t1-lag topology.
./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt
Load minigraph/config_dbs.
Ensure all internal and external interfaces come up.
No change on single asic vs.

- Which release branch to backport (provide reason below if selected)

201811
201911
202006
202012

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

a four asic hwsku with 8 front panel interfaces. This modification is done to keep the virtual switch lighter than current six asic hwsku. Signed-off-by: SuvarnaMeenakshi <[email protected]>

8 interfaces virtual switch instead of 64 interfaces. Signed-off-by: SuvarnaMeenakshi <[email protected]>

arlakshm · 2021-01-25T20:28:49Z

device/virtual/x86_64-kvm_x86_64-r0/msft_multi_asic_vs/topology_6_asic.sh

@@ -0,0 +1,82 @@
+#!/bin/bash


I think we can remove this file

Removed this file.

Signed-off-by: SuvarnaMeenakshi <[email protected]>

arlakshm · 2021-01-25T20:39:06Z

device/virtual/x86_64-kvm_x86_64-r0/msft_multi_asic_vs/topology.sh

            sudo ip netns exec asic$ASIC ip link set dev eth$NUM down
            sudo ip netns exec asic$ASIC ip link delete dev eth$NUM
+            echo "$ASIC: eth$NUM"


do we need this echo statement?

We don't need this echo statement, removed echo statements added for debugging.

Signed-off-by: SuvarnaMeenakshi <[email protected]>

judyjoseph · 2021-02-02T07:24:01Z

LGTM

Keep the msft_multi_asic vs hwsku with 6 asics configuration. Signed-off-by: SuvarnaMeenakshi <[email protected]>

judyjoseph

Also to confirm the plan is to use the file sonic_multiasic.xml for both 4 and 6 ASIC vs

SuvarnaMeenakshi · 2021-02-22T23:16:06Z

Also to confirm the plan is to use the file sonic_multiasic.xml for both 4 and 6 ASIC vs

sonic_mutliasic.xml in platform/vs directory supports 6 ASIC vs with 64 interfaces.
If we have to bring up 4 ASIC vs, this file has to be modified to have 8 external interfaces.
I have mentioned that point in the PR description, I will raise a separate PR to update README file to bring up 6-ASIC or 4-ASIC VS.

In sonic-mgmt, the xml file to create KVM is generated based on the number of front-panel interfaces.

Thanks, good to update the README.

SuvarnaMeenakshi · 2021-02-23T21:35:38Z

Also to confirm the plan is to use the file sonic_multiasic.xml for both 4 and 6 ASIC vs

sonic_mutliasic.xml in platform/vs directory supports 6 ASIC vs with 64 interfaces.
If we have to bring up 4 ASIC vs, this file has to be modified to have 8 external interfaces.
I have mentioned that point in the PR description, I will raise a separate PR to update README file to bring up 6-ASIC or 4-ASIC VS.

In sonic-mgmt, the xml file to create KVM is generated based on the number of front-panel interfaces.

Thanks, good to update the README.

Have updated README here: #6867

…d 8 interfaces (#2858) What is the motivation for this PR? sonic-net/sonic-buildimage#6558 will add new hwsku msft_four_asic_vs which is a 4 asic hwsku with 8 front panel interfaces. With this change, add supporting files to sonic-mgmt so that the new 4 asic hwsku testbed can be brought up. How did you do it? Add new t1-lag topo file to support 8 front panel interfaces. Modify sonic.xml.j2 file to use higher number of vcpus based on the multi-asic hwsku, 6 asic hwsku will require more vcpus. Add new testbed for msft_four_asic_vs and new DUT vlab-08. Add a new minigraph for mutli-asic vlab-08 DUT to come up with t1-lag topology. Start topology service before load_minigraph for multi-asic vs platform. How did you verify/test it? Bring up multi-asic vs image with 4-asics testbed using add-topo: ./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt Copy the minigraph to virtual switch. Start topology service (systemctl start topology.service) load minigraph. Ensure that all internal and external interfaces are up and all BGP sessions are up.

- Why I did it Current mutli-asic vs hwsku consists of 6 asics with each asic having 32 interfaces. When bringing this up, below issue was seen: When all 32 interfaces(sonic interfaces and linux interface) are set to 9100 mtu, DMA error is seen "DMA: Out of SW-IOMMU space for 4096 bytes at device 0000:06:03.0" which can be fixed by updating swiotlb=65536 in /host/grub/grub.cfg .In order to keep multi-asic VS lighter and easier to bring up and test, new hwsku 'msft_four_asic_vs' is added to represent 4-asic hwsku with 2 frontend asics and 2 backend asics and each asic having 8 interfaces interconnected by port-channels. - How I did it Add msft_four_asic_hwsku directory to have the right number of directories (4) and update port_config.ini and lanemap.ini files to include 8 ports information. Add topology.sh script to create the internal asic-asic connectivity. - How to verify it Update asic.conf with the 4 asic information as below and build sonic-vs.img: NUM_ASIC=4 DEV_ID_ASIC_0=0 DEV_ID_ASIC_1=1 DEV_ID_ASIC_2=2 DEV_ID_ASIC_3=3 Modify sonic_multiasic.xml to have 8 front panel interfaces. create virtual switch using "sudo virsh sonic_mutliasic.xml" command. Start topology service and Load config_db files for switch and each asic. Ensure that that all internal interfaces and port_channels are coming up. multi-asic vs testbed: Bring up mutli-asic VS testbed with a multi-asic image(asic.conf updated to 4 asics) and using t1-lag topology. ./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt Load minigraph/config_dbs. Ensure all internal and external interfaces come up. No change on single asic vs.

…net#6558) - Why I did it Current mutli-asic vs hwsku consists of 6 asics with each asic having 32 interfaces. When bringing this up, below issue was seen: When all 32 interfaces(sonic interfaces and linux interface) are set to 9100 mtu, DMA error is seen "DMA: Out of SW-IOMMU space for 4096 bytes at device 0000:06:03.0" which can be fixed by updating swiotlb=65536 in /host/grub/grub.cfg .In order to keep multi-asic VS lighter and easier to bring up and test, new hwsku 'msft_four_asic_vs' is added to represent 4-asic hwsku with 2 frontend asics and 2 backend asics and each asic having 8 interfaces interconnected by port-channels. - How I did it Add msft_four_asic_hwsku directory to have the right number of directories (4) and update port_config.ini and lanemap.ini files to include 8 ports information. Add topology.sh script to create the internal asic-asic connectivity. - How to verify it Update asic.conf with the 4 asic information as below and build sonic-vs.img: NUM_ASIC=4 DEV_ID_ASIC_0=0 DEV_ID_ASIC_1=1 DEV_ID_ASIC_2=2 DEV_ID_ASIC_3=3 Modify sonic_multiasic.xml to have 8 front panel interfaces. create virtual switch using "sudo virsh sonic_mutliasic.xml" command. Start topology service and Load config_db files for switch and each asic. Ensure that that all internal interfaces and port_channels are coming up. multi-asic vs testbed: Bring up mutli-asic VS testbed with a multi-asic image(asic.conf updated to 4 asics) and using t1-lag topology. ./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt Load minigraph/config_dbs. Ensure all internal and external interfaces come up. No change on single asic vs.

SuvarnaMeenakshi added 2 commits January 25, 2021 11:34

[multi-asic][vs]: Modify multi-asic vs hwsku 'msft_multi_asic_vs' to be

6b2d28b

a four asic hwsku with 8 front panel interfaces. This modification is done to keep the virtual switch lighter than current six asic hwsku. Signed-off-by: SuvarnaMeenakshi <[email protected]>

[multi-asic][vs]: Modify sonic_multiasic.xml to come up as

ce2c0b9

8 interfaces virtual switch instead of 64 interfaces. Signed-off-by: SuvarnaMeenakshi <[email protected]>

SuvarnaMeenakshi requested review from judyjoseph, arlakshm and abdosi January 25, 2021 20:08

SuvarnaMeenakshi mentioned this pull request Jan 25, 2021

[mutli-asic][vs]: Add support to test multi-asic vs with four asic and 8 interfaces sonic-net/sonic-mgmt#2858

Merged

3 tasks

arlakshm reviewed Jan 25, 2021

View reviewed changes

Remove topology script for 6 asic virtual switch.

8de25c4

Signed-off-by: SuvarnaMeenakshi <[email protected]>

arlakshm reviewed Jan 25, 2021

View reviewed changes

SuvarnaMeenakshi added 2 commits January 25, 2021 13:03

[topology]: Remove debug echo statements.

a71c2bf

Signed-off-by: SuvarnaMeenakshi <[email protected]>

[topology]: Removed duplicate line.

7e21786

Signed-off-by: SuvarnaMeenakshi <[email protected]>

SuvarnaMeenakshi requested review from jleveque and lguohan as code owners February 6, 2021 20:29

Add a new multi-asic vs hwsku with 4 asics.

a628899

Keep the msft_multi_asic vs hwsku with 6 asics configuration. Signed-off-by: SuvarnaMeenakshi <[email protected]>

SuvarnaMeenakshi changed the title ~~[multi-asic][vs]: Modify multi-asic vs hwsku 'msft_multi_asic_vs' to four asic~~ [multi-asic][vs]: Add new multi-asic vs hwsku with four asics Feb 16, 2021

arlakshm approved these changes Feb 19, 2021

View reviewed changes

judyjoseph approved these changes Feb 22, 2021

View reviewed changes

SuvarnaMeenakshi added the Request for 201911 Branch label Feb 22, 2021

SuvarnaMeenakshi merged commit d88e8cf into sonic-net:master Feb 23, 2021

abdosi added the Included in 201911 Branch label Feb 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[multi-asic][vs]: Add new multi-asic vs hwsku with four asics #6558

[multi-asic][vs]: Add new multi-asic vs hwsku with four asics #6558

SuvarnaMeenakshi commented Jan 25, 2021 •

edited

Loading

arlakshm Jan 25, 2021

SuvarnaMeenakshi Jan 25, 2021

arlakshm Jan 25, 2021

SuvarnaMeenakshi Jan 25, 2021

judyjoseph commented Feb 2, 2021

judyjoseph left a comment •

edited

Loading

SuvarnaMeenakshi commented Feb 22, 2021 •

edited by judyjoseph

Loading

SuvarnaMeenakshi commented Feb 23, 2021

[multi-asic][vs]: Add new multi-asic vs hwsku with four asics #6558

[multi-asic][vs]: Add new multi-asic vs hwsku with four asics #6558

Conversation

SuvarnaMeenakshi commented Jan 25, 2021 • edited Loading

arlakshm Jan 25, 2021

Choose a reason for hiding this comment

SuvarnaMeenakshi Jan 25, 2021

Choose a reason for hiding this comment

arlakshm Jan 25, 2021

Choose a reason for hiding this comment

SuvarnaMeenakshi Jan 25, 2021

Choose a reason for hiding this comment

judyjoseph commented Feb 2, 2021

judyjoseph left a comment • edited Loading

Choose a reason for hiding this comment

SuvarnaMeenakshi commented Feb 22, 2021 • edited by judyjoseph Loading

SuvarnaMeenakshi commented Feb 23, 2021

SuvarnaMeenakshi commented Jan 25, 2021 •

edited

Loading

judyjoseph left a comment •

edited

Loading

SuvarnaMeenakshi commented Feb 22, 2021 •

edited by judyjoseph

Loading