scrape jobs never close connection

[helgharbi]

We received errors Call to the Redfish API failed (RedfishCallFailed)

in the log we have

INFO Login returned code 401: {"error":{"@Message.ExtendedInfo":[{"Message":"Unable to complete the operation because an invalid username and/or password is entered, and therefore authentication failed.","MessageArgs":[],"[email protected]":0,"MessageId":"IDRAC.2.5.SYS415","RelatedProperties":[],"[email protected]":0,"Resolution":"Enter valid user name and password and retry the operation.","Severity":"Warning"}],"code":"Base.1.8.GeneralError","message":"A general error has occurred. See ExtendedInfo for more information"}}

After investigation we found that Redfish connection to iDRAC never ends causing new connection slow to establish.

racadm getssninfo
SSNID Type              User                                IP Address Login Date/Time    
---------------------------------------------------------------------------
249   REDFISH           root                               a.b.c.d 12/11/2023 11:40:25
250   REDFISH           root                               a.b.c.d 12/11/2023 11:40:33
254   REDFISH           root                               a.b.c.d 12/11/2023 11:46:25
255   REDFISH           root                               a.b.c.d 12/11/2023 11:46:33
257   REDFISH           root                               a.b.c.d12/11/2023 11:47:37
258   REDFISH           root                               a.b.c.d 12/11/2023 11:47:39
261   REDFISH           root                               a.b.c.d 12/11/2023 11:53:24
262   REDFISH           root                               a.b.c.d 12/11/2023 11:53:31

we found a workaround by running racadm racreset soft but the problem keeps coming back.

[przemeklal]

Possibly related to canonical/prometheus-hardware-exporter#48

[chanchiwai-ray]

Are you able to provide more logs from the exporter service, or from get more information racadm cli to help us troubleshoot the issue? From the source code where the redfish login and logout happens, we can't identity why it's being happening; there are logout command accompanied by login command...

[przemeklal]

Shared internally.

[chanchiwai-ray]

The PR I link can potentially reduce the occurrence of this (or even fix the issue altogether) if it's caused by the case you shared.