Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[warm-reboot] Fix key error - check empty key before issuing redis hget #1496

Merged
merged 1 commit into from
Mar 12, 2021

Conversation

vaibhavhd
Copy link
Contributor

What I did

Fix for: sonic-net/sonic-buildimage#7015

How I did it

Added a check for empty key before checking WARM_RESTART_ENABLE_TABLE

How to verify it

Without fix:

admin@str-s6100-acs-2:~$ sudo warm-reboot -vvv
Traceback (most recent call last):
  File "/usr/local/bin/sonic-db-cli", line 131, in <module>
    main()
  File "/usr/local/bin/sonic-db-cli", line 114, in main
    execute_cmd(args.db_or_op, args.cmd, args.namespace, args.unixsocket)
  File "/usr/local/bin/sonic-db-cli", line 73, in execute_cmd
    resp = client.execute_command(*cmd)
  File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 901, in execute_command
    return self.parse_response(conn, command_name, **options)
  File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 915, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 756, in read_response
    raise response
redis.exceptions.ResponseError: wrong number of arguments for 'hget' command

With fix (after cold reboot, issue a warm-reboot):

admin@str-s6100-acs-2:~$ sonic-db-cli STATE_DB hget "WARM_RESTART_ENABLE_TABLE|system" enable

admin@str-s6100-acs-2:~$ sonic-db-cli STATE_DB keys "WARM_RESTART_ENABLE_TABLE|*"            

admin@str-s6100-acs-2:~$ sudo warm-reboot -vvv
Thu 11 Mar 2021 08:41:48 PM UTC Saving counters folder before warmboot...
Thu 11 Mar 2021 08:41:51 PM UTC Pausing orchagent ...
Thu 11 Mar 2021 08:41:51 PM UTC Collecting logs to check ssd health before warm-reboot...
Thu 11 Mar 2021 08:41:51 PM UTC Stopping nat ...
Dumping conntrack entries failed
Error response from daemon: Cannot kill container: nat: No such container: nat
Thu 11 Mar 2021 08:41:51 PM UTC Stopped nat ...
Thu 11 Mar 2021 08:41:51 PM UTC Stopping radv service...
Thu 11 Mar 2021 08:41:52 PM UTC Stopped radv service...
Thu 11 Mar 2021 08:41:52 PM UTC Stopping bgp ...
Thu 11 Mar 2021 08:41:57 PM UTC Stopped bgp ...
Thu 11 Mar 2021 08:41:57 PM UTC Stopping sflow ...
Thu 11 Mar 2021 08:41:58 PM UTC Stopped sflow ...
Thu 11 Mar 2021 08:42:00 PM UTC Stopping swss service ...
Thu 11 Mar 2021 08:42:01 PM UTC Stopped swss service ...
Thu 11 Mar 2021 08:42:01 PM UTC Initialize pre-shutdown ...
Thu 11 Mar 2021 08:42:02 PM UTC Requesting pre-shutdown ...
Thu 11 Mar 2021 08:42:02 PM UTC Waiting for pre-shutdown ...
Thu 11 Mar 2021 08:42:03 PM UTC Pre-shutdown succeeded ...
Thu 11 Mar 2021 08:42:03 PM UTC Backing up database ...
Thu 11 Mar 2021 08:42:04 PM UTC Stopping teamd ...
Thu 11 Mar 2021 08:42:10 PM UTC Stopped teamd ...
Thu 11 Mar 2021 08:42:10 PM UTC Stopping syncd ...
Thu 11 Mar 2021 08:42:26 PM UTC Stopped syncd ...
Thu 11 Mar 2021 08:42:27 PM UTC Stopping all remaining containers ...
Thu 11 Mar 2021 08:42:35 PM UTC Stopped all remaining containers ...
Thu 11 Mar 2021 08:42:37 PM UTC updating ssd fw forwarm-reboot
Thu 11 Mar 2021 08:42:37 PM UTC Enabling Watchdog before warm-reboot
Watchdog armed for 180 seconds
Thu 11 Mar 2021 08:42:38 PM UTC Running x86_64-dell_s6100_c2538-r0 specific plugin...
Thu 11 Mar 2021 08:42:38 PM UTC Rebooting with /sbin/kexec -e to SONiC-OS-HEAD.487-377ea1a2 ...

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

@vaibhavhd vaibhavhd added the Bug label Mar 11, 2021
@vaibhavhd vaibhavhd requested review from yxieca and shi-su March 11, 2021 20:47
@vaibhavhd vaibhavhd self-assigned this Mar 11, 2021
@vaibhavhd
Copy link
Contributor Author

@shi-su was the change in #1460 included in 202012 branch?
If yes, we need to add this fix to 202012 too. I did not see a label for 2020 in the PR 1460, but could not find a submodule update PR for 1460.

@shi-su
Copy link
Contributor

shi-su commented Mar 12, 2021

@shi-su was the change in #1460 included in 202012 branch?
If yes, we need to add this fix to 202012 too. I did not see a label for 2020 in the PR 1460, but could not find a submodule update PR for 1460.

I do think we need # 1460 for 202012 branch. It is not included yet. I added the label to request it.

@vaibhavhd
Copy link
Contributor Author

retest this please

@lguohan lguohan merged commit 1260f90 into sonic-net:master Mar 12, 2021
@vaibhavhd vaibhavhd deleted the warmboot-fix-key-err branch March 12, 2021 17:27
daall pushed a commit that referenced this pull request Mar 12, 2021
Added a check for empty key before checking WARM_RESTART_ENABLE_TABLE
stepanblyschak pushed a commit to stepanblyschak/sonic-utilities that referenced this pull request Apr 28, 2022
Including the following changes:
[bitmap_vnet] Remove BMTOR implementation (sonic-net#1496)
[intfsorch] Init proxy_arp variable while adding router interface. (sonic-net#1473)
[drop counters] Clarify log messages for initial counter setup (sonic-net#1445)

Signed-off-by: Liat Grozovik <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants