-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[multi-asic][vs]: Add new multi-asic vs hwsku with four asics (#6558)
- Why I did it Current mutli-asic vs hwsku consists of 6 asics with each asic having 32 interfaces. When bringing this up, below issue was seen: When all 32 interfaces(sonic interfaces and linux interface) are set to 9100 mtu, DMA error is seen "DMA: Out of SW-IOMMU space for 4096 bytes at device 0000:06:03.0" which can be fixed by updating swiotlb=65536 in /host/grub/grub.cfg .In order to keep multi-asic VS lighter and easier to bring up and test, new hwsku 'msft_four_asic_vs' is added to represent 4-asic hwsku with 2 frontend asics and 2 backend asics and each asic having 8 interfaces interconnected by port-channels. - How I did it Add msft_four_asic_hwsku directory to have the right number of directories (4) and update port_config.ini and lanemap.ini files to include 8 ports information. Add topology.sh script to create the internal asic-asic connectivity. - How to verify it Update asic.conf with the 4 asic information as below and build sonic-vs.img: NUM_ASIC=4 DEV_ID_ASIC_0=0 DEV_ID_ASIC_1=1 DEV_ID_ASIC_2=2 DEV_ID_ASIC_3=3 Modify sonic_multiasic.xml to have 8 front panel interfaces. create virtual switch using "sudo virsh sonic_mutliasic.xml" command. Start topology service and Load config_db files for switch and each asic. Ensure that that all internal interfaces and port_channels are coming up. multi-asic vs testbed: Bring up mutli-asic VS testbed with a multi-asic image(asic.conf updated to 4 asics) and using t1-lag topology. ./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt Load minigraph/config_dbs. Ensure all internal and external interfaces come up. No change on single asic vs.
- Loading branch information
1 parent
4ddf3fc
commit d88e8cf
Showing
41 changed files
with
3,373 additions
and
0 deletions.
There are no files selected for viewing
3 changes: 3 additions & 0 deletions
3
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/buffers.json.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| {%- set default_topo = 't1' %} | ||
| {%- include 'buffers_config.j2' %} | ||
|
|
45 changes: 45 additions & 0 deletions
45
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/buffers_defaults_def.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| {%- set default_cable = '300m' %} | ||
|
|
||
| {%- macro generate_port_lists(PORT_ALL) %} | ||
| {# Generate list of ports #} | ||
| {% for port_idx in range(0,32) %} | ||
| {% if PORT_ALL.append("Ethernet%d" % (port_idx * 4)) %}{% endif %} | ||
| {% endfor %} | ||
| {%- endmacro %} | ||
|
|
||
| {%- macro generate_buffer_pool_and_profiles() %} | ||
| "BUFFER_POOL": { | ||
| "ingress_lossless_pool": { | ||
| "size": "12766208", | ||
| "type": "ingress", | ||
| "mode": "dynamic" | ||
| }, | ||
| "egress_lossless_pool": { | ||
| "size": "12766208", | ||
| "type": "egress", | ||
| "mode": "static" | ||
| }, | ||
| "egress_lossy_pool": { | ||
| "size": "7326924", | ||
| "type": "egress", | ||
| "mode": "dynamic" | ||
| } | ||
| }, | ||
| "BUFFER_PROFILE": { | ||
| "ingress_lossy_profile": { | ||
| "pool":"[BUFFER_POOL|ingress_lossless_pool]", | ||
| "size":"0", | ||
| "dynamic_th":"3" | ||
| }, | ||
| "egress_lossless_profile": { | ||
| "pool":"[BUFFER_POOL|egress_lossless_pool]", | ||
| "size":"0", | ||
| "static_th":"12766208" | ||
| }, | ||
| "egress_lossy_profile": { | ||
| "pool":"[BUFFER_POOL|egress_lossy_pool]", | ||
| "size":"1518", | ||
| "dynamic_th":"3" | ||
| } | ||
| }, | ||
| {%- endmacro %} |
45 changes: 45 additions & 0 deletions
45
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/buffers_defaults_t0.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| {%- set default_cable = '300m' %} | ||
|
|
||
| {%- macro generate_port_lists(PORT_ALL) %} | ||
| {# Generate list of ports #} | ||
| {% for port_idx in range(0,32) %} | ||
| {% if PORT_ALL.append("Ethernet%d" % (port_idx * 4)) %}{% endif %} | ||
| {% endfor %} | ||
| {%- endmacro %} | ||
|
|
||
| {%- macro generate_buffer_pool_and_profiles() %} | ||
| "BUFFER_POOL": { | ||
| "ingress_lossless_pool": { | ||
| "size": "12766208", | ||
| "type": "ingress", | ||
| "mode": "dynamic" | ||
| }, | ||
| "egress_lossless_pool": { | ||
| "size": "12766208", | ||
| "type": "egress", | ||
| "mode": "static" | ||
| }, | ||
| "egress_lossy_pool": { | ||
| "size": "7326924", | ||
| "type": "egress", | ||
| "mode": "dynamic" | ||
| } | ||
| }, | ||
| "BUFFER_PROFILE": { | ||
| "ingress_lossy_profile": { | ||
| "pool":"[BUFFER_POOL|ingress_lossless_pool]", | ||
| "size":"0", | ||
| "dynamic_th":"3" | ||
| }, | ||
| "egress_lossless_profile": { | ||
| "pool":"[BUFFER_POOL|egress_lossless_pool]", | ||
| "size":"0", | ||
| "static_th":"12766208" | ||
| }, | ||
| "egress_lossy_profile": { | ||
| "pool":"[BUFFER_POOL|egress_lossy_pool]", | ||
| "size":"1518", | ||
| "dynamic_th":"3" | ||
| } | ||
| }, | ||
| {%- endmacro %} |
45 changes: 45 additions & 0 deletions
45
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/buffers_defaults_t1.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| {%- set default_cable = '300m' %} | ||
|
|
||
| {%- macro generate_port_lists(PORT_ALL) %} | ||
| {# Generate list of ports #} | ||
| {% for port_idx in range(0,32) %} | ||
| {% if PORT_ALL.append("Ethernet%d" % (port_idx * 4)) %}{% endif %} | ||
| {% endfor %} | ||
| {%- endmacro %} | ||
|
|
||
| {%- macro generate_buffer_pool_and_profiles() %} | ||
| "BUFFER_POOL": { | ||
| "ingress_lossless_pool": { | ||
| "size": "12766208", | ||
| "type": "ingress", | ||
| "mode": "dynamic" | ||
| }, | ||
| "egress_lossless_pool": { | ||
| "size": "12766208", | ||
| "type": "egress", | ||
| "mode": "static" | ||
| }, | ||
| "egress_lossy_pool": { | ||
| "size": "7326924", | ||
| "type": "egress", | ||
| "mode": "dynamic" | ||
| } | ||
| }, | ||
| "BUFFER_PROFILE": { | ||
| "ingress_lossy_profile": { | ||
| "pool":"[BUFFER_POOL|ingress_lossless_pool]", | ||
| "size":"0", | ||
| "dynamic_th":"3" | ||
| }, | ||
| "egress_lossless_profile": { | ||
| "pool":"[BUFFER_POOL|egress_lossless_pool]", | ||
| "size":"0", | ||
| "static_th":"12766208" | ||
| }, | ||
| "egress_lossy_profile": { | ||
| "pool":"[BUFFER_POOL|egress_lossy_pool]", | ||
| "size":"1518", | ||
| "dynamic_th":"3" | ||
| } | ||
| }, | ||
| {%- endmacro %} |
8 changes: 8 additions & 0 deletions
8
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/lanemap.ini
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| eth1:1,2,3,4 | ||
| eth2:5,6,7,8 | ||
| eth3:9,10,11,12 | ||
| eth4:13,14,15,16 | ||
| eth5:17,18,19,20 | ||
| eth6:21,22,23,24 | ||
| eth7:25,26,27,28 | ||
| eth8:29,30,31,32 |
17 changes: 17 additions & 0 deletions
17
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/pg_profile_lookup.ini
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # PG lossless profiles. | ||
| # speed cable size xon xoff threshold xon_offset | ||
| 10000 5m 56368 18432 55120 -3 2496 | ||
| 25000 5m 56368 18432 55120 -3 2496 | ||
| 40000 5m 56368 18432 55120 -3 2496 | ||
| 50000 5m 56368 18432 55120 -3 2496 | ||
| 100000 5m 56368 18432 55120 -3 2496 | ||
| 10000 40m 56368 18432 55120 -3 2496 | ||
| 25000 40m 56368 18432 55120 -3 2496 | ||
| 40000 40m 56368 18432 55120 -3 2496 | ||
| 50000 40m 56368 18432 55120 -3 2496 | ||
| 100000 40m 56368 18432 55120 -3 2496 | ||
| 10000 300m 56368 18432 55120 -3 2496 | ||
| 25000 300m 56368 18432 55120 -3 2496 | ||
| 40000 300m 56368 18432 55120 -3 2496 | ||
| 50000 300m 56368 18432 55120 -3 2496 | ||
| 100000 300m 56368 18432 55120 -3 2496 |
9 changes: 9 additions & 0 deletions
9
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/port_config.ini
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # name lanes alias index asic_port_name role | ||
| Ethernet0 1,2,3,4 Ethernet1/1 0 Eth0-ASIC0 Ext | ||
| Ethernet4 5,6,7,8 Ethernet1/2 1 Eth1-ASIC0 Ext | ||
| Ethernet8 9,10,11,12 Ethernet1/3 2 Eth2-ASIC0 Ext | ||
| Ethernet12 13,14,15,16 Ethernet1/4 3 Eth3-ASIC0 Ext | ||
| Ethernet-BP0 17,18,19,20 Eth4-ASIC0 0 Eth4-ASIC0 Int | ||
| Ethernet-BP4 21,22,23,24 Eth5-ASIC0 1 Eth5-ASIC0 Int | ||
| Ethernet-BP8 25,26,27,28 Eth6-ASIC0 2 Eth6-ASIC0 Int | ||
| Ethernet-BP12 29,30,31,32 Eth7-ASIC0 3 Eth7-ASIC0 Int |
1 change: 1 addition & 0 deletions
1
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/qos.json.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| {%- include 'qos_config.j2' %} |
3 changes: 3 additions & 0 deletions
3
device/virtual/x86_64-kvm_x86_64-r0/msft_four_asic_vs/0/sai.profile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| SAI_VS_SWITCH_TYPE=SAI_VS_SWITCH_TYPE_BCM56850 | ||
| SAI_VS_HOSTIF_USE_TAP_DEVICE=true | ||
| SAI_VS_INTERFACE_LANE_MAP_FILE=/usr/share/sonic/hwsku/lanemap.ini |
Oops, something went wrong.