No link with AS7816-64X

[titanlock]

As the title states I cannot get any link to happen between my switch and a host. I have tried changing MTUs and speeds but no link. The cables are good and would get instant link from one host to another. I am even able to see the eeprom info of the cable on the switch.

Also I had made a build from the master branch, there was no official build for this device. Not sure if this is the main cause to my issues or not.

Here is some info about the switch:

admin@intel-edgecore:~$ show interfaces status 
  Interface            Lanes    Speed    MTU          Alias      Vlan    Oper    Admin             Type    Asym PFC
-----------  ---------------  -------  -----  -------------  --------  ------  -------  ---------------  ----------
  Ethernet0      73,74,75,76     100G   9100   hundredGigE1  VLAN3008     N/A       up  QSFP28 or later         N/A
  Ethernet4      65,66,67,68     100G   9100   hundredGigE2  VLAN3008     N/A       up  QSFP28 or later         N/A
  Ethernet8      81,82,83,84     100G   9100   hundredGigE3  VLAN3008     N/A       up  QSFP28 or later         N/A
 Ethernet12      89,90,91,92     100G   9100   hundredGigE4  VLAN3008     N/A       up  QSFP28 or later         N/A
 Ethernet16  105,106,107,108     100G   9100   hundredGigE5  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet20     97,98,99,100      40G   1500   hundredGigE6  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet24  113,114,115,116     100G   9100   hundredGigE7  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet28  121,122,123,124     100G   9100   hundredGigE8  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet32      41,42,43,44     100G   9100   hundredGigE9  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet36      33,34,35,36     100G   9100  hundredGigE10  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet40      49,50,51,52     100G   9100  hundredGigE11  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet44      57,58,59,60     100G   9100  hundredGigE12  VLAN3008     N/A       up   QSFP+ or later         N/A
 Ethernet48  137,138,139,140     100G   9100  hundredGigE13    routed     N/A       up              N/A         N/A
 Ethernet52  129,130,131,132     100G   9100  hundredGigE14    routed     N/A       up              N/A         N/A
 Ethernet56  145,146,147,148     100G   9100  hundredGigE15    routed     N/A       up              N/A         N/A
 Ethernet60  153,154,155,156     100G   9100  hundredGigE16    routed     N/A       up              N/A         N/A
 Ethernet64  173,174,175,176     100G   9100  hundredGigE17    routed     N/A       up              N/A         N/A
 Ethernet68  165,166,167,168     100G   9100  hundredGigE18    routed     N/A       up              N/A         N/A
 Ethernet72  181,182,183,184     100G   9100  hundredGigE19    routed     N/A       up              N/A         N/A
 Ethernet76  189,190,191,192     100G   9100  hundredGigE20    routed     N/A       up              N/A         N/A
 Ethernet80      13,14,15,16     100G   9100  hundredGigE21    routed     N/A       up              N/A         N/A
 Ethernet84          5,6,7,8     100G   9100  hundredGigE22    routed     N/A       up              N/A         N/A
 Ethernet88      29,30,31,32     100G   9100  hundredGigE23    routed     N/A       up              N/A         N/A
 Ethernet92      21,22,23,24     100G   9100  hundredGigE24    routed     N/A       up              N/A         N/A
 Ethernet96  205,206,207,208     100G   9100  hundredGigE25    routed     N/A       up              N/A         N/A
Ethernet100  197,198,199,200     100G   9100  hundredGigE26    routed     N/A       up              N/A         N/A
Ethernet104  213,214,215,216     100G   9100  hundredGigE27    routed     N/A       up              N/A         N/A
Ethernet108  221,222,223,224     100G   9100  hundredGigE28    routed     N/A       up              N/A         N/A
Ethernet112  229,230,231,232     100G   9100  hundredGigE29    routed     N/A       up              N/A         N/A
Ethernet116  237,238,239,240     100G   9100  hundredGigE30    routed     N/A       up              N/A         N/A
Ethernet120  245,246,247,248     100G   9100  hundredGigE31  VLAN3010     N/A       up   QSFP+ or later         N/A
Ethernet124  253,254,255,256     100G   9100  hundredGigE32    routed     N/A       up              N/A         N/A
Ethernet128      69,70,71,72     100G   9100  hundredGigE33    routed     N/A       up              N/A         N/A
Ethernet132      77,78,79,80     100G   9100  hundredGigE34    routed     N/A       up              N/A         N/A
Ethernet136      93,94,95,96     100G   9100  hundredGigE35    routed     N/A       up              N/A         N/A
Ethernet140      85,86,87,88     100G   9100  hundredGigE36    routed     N/A       up              N/A         N/A
Ethernet144  101,102,103,104     100G   9100  hundredGigE37    routed     N/A       up              N/A         N/A
Ethernet148  109,110,111,112     100G   9100  hundredGigE38    routed     N/A       up              N/A         N/A
Ethernet152  125,126,127,128     100G   9100  hundredGigE39    routed     N/A       up              N/A         N/A
Ethernet156  117,118,119,120     100G   9100  hundredGigE40    routed     N/A       up              N/A         N/A
Ethernet160      37,38,39,40     100G   9100  hundredGigE41    routed     N/A       up              N/A         N/A
Ethernet164      45,46,47,48     100G   9100  hundredGigE42    routed     N/A       up              N/A         N/A
Ethernet168      61,62,63,64     100G   9100  hundredGigE43    routed     N/A       up              N/A         N/A
Ethernet172      53,54,55,56     100G   9100  hundredGigE44    routed     N/A       up              N/A         N/A
Ethernet176  133,134,135,136     100G   9100  hundredGigE45    routed     N/A       up              N/A         N/A
Ethernet180  141,142,143,144     100G   9100  hundredGigE46    routed     N/A       up              N/A         N/A
Ethernet184  157,158,159,160     100G   9100  hundredGigE47    routed     N/A       up              N/A         N/A
Ethernet188  149,150,151,152     100G   9100  hundredGigE48    routed     N/A       up              N/A         N/A
Ethernet192  161,162,163,164     100G   9100  hundredGigE49    routed     N/A       up              N/A         N/A
Ethernet196  169,170,171,172     100G   9100  hundredGigE50    routed     N/A       up              N/A         N/A
Ethernet200  185,186,187,188     100G   9100  hundredGigE51    routed     N/A       up   QSFP+ or later         N/A
Ethernet204  177,178,179,180     100G   9100  hundredGigE52    routed     N/A       up              N/A         N/A
Ethernet208          1,2,3,4     100G   9100  hundredGigE53    routed     N/A       up              N/A         N/A
Ethernet212       9,10,11,12     100G   9100  hundredGigE54    routed     N/A       up              N/A         N/A
Ethernet216      25,26,27,28     100G   9100  hundredGigE55    routed     N/A       up              N/A         N/A
Ethernet220      17,18,19,20     100G   9100  hundredGigE56    routed     N/A       up              N/A         N/A
Ethernet224  193,194,195,196     100G   9100  hundredGigE57    routed     N/A       up              N/A         N/A
Ethernet228  201,202,203,204     100G   9100  hundredGigE58    routed     N/A       up              N/A         N/A
Ethernet232  217,218,219,220     100G   9100  hundredGigE59    routed     N/A       up              N/A         N/A
Ethernet236  209,210,211,212     100G   9100  hundredGigE60    routed     N/A       up              N/A         N/A
Ethernet240  225,226,227,228     100G   9100  hundredGigE61    routed     N/A       up              N/A         N/A
Ethernet244  233,234,235,236     100G   9100  hundredGigE62    routed     N/A       up              N/A         N/A
Ethernet248  249,250,251,252     100G   9100  hundredGigE63    routed     N/A       up              N/A         N/A
Ethernet252  241,242,243,244     100G   9100  hundredGigE64    routed     N/A       up              N/A         N/A

admin@intel-edgecore:~$ docker ps -a
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS                    PORTS               NAMES
8d0b82d33562        docker-syncd-brcm:latest          "/usr/bin/supervisord"   12 days ago         Up 3 minutes                                  syncd
1a8e6c510019        docker-dhcp-relay:latest          "/usr/bin/docker_ini…"   12 days ago         Exited (0) 5 days ago                         dhcp_relay
a710f3a881da        docker-snmp-sv2:latest            "/usr/bin/supervisord"   12 days ago         Exited (137) 5 days ago                       snmp
8e128275ab7f        docker-sflow:latest               "/usr/bin/supervisord"   12 days ago         Exited (0) 12 days ago                        sflow
033184f5085b        docker-router-advertiser:latest   "/usr/bin/supervisord"   12 days ago         Exited (0) 5 days ago                         radv
d2aeab6e2621        docker-teamd:latest               "/usr/bin/supervisord"   12 days ago         Up 5 days                                     teamd
e39e296df2e3        docker-platform-monitor:latest    "/usr/bin/docker_ini…"   12 days ago         Up 5 days                                     pmon
90827a852c2e        docker-orchagent:latest           "/usr/bin/supervisord"   12 days ago         Exited (0) 5 days ago                         swss
972e9c1c01ed        docker-fpm-frr:latest             "/usr/bin/supervisord"   12 days ago         Up 5 days                                     bgp
cfa1675437b8        docker-lldp-sv2:latest            "/usr/bin/supervisord"   12 days ago         Up 5 days                                     lldp
1e69266e0327        docker-sonic-telemetry:latest     "/usr/bin/supervisord"   12 days ago         Up 12 days                                    telemetry
e66d42115af1        docker-database:latest            "/usr/local/bin/dock…"   2 weeks ago         Up 12 days                                    database

Any info will be helpful.

Thanks

[pollyhsu2git]

@titanlock
For the AS7816-64X issue on the SONiC latest build we found, we've high-lighted its syncd crashed via sonic-net/sonic-buildimage#3508. When the SONiC syncd crashes, you would not see any port link.

[titanlock]

@pollyhsu2git Thank you for the reference and guide, with your workaround I am able to get full link on my Edgecore switches now.

You are a life saver.