openbmc-tools icon indicating copy to clipboard operation
openbmc-tools copied to clipboard

System encountered a BMC error that policy table could not understand

Open yesyjim opened this issue 5 years ago • 4 comments

I think we have an issue with the policy table or missing information to interpret.

Customer system got a SEL event:

12535 | Unknown | 2020-02-11 13:56:24 | Unknown | Critical | error: Not found in policy table: Sensor.Device.Error.ReadFailure||

But it really is not telling us what sensor. When I go into the elog file I see the following for this event:

DICT_ENTRY "oa{sa{sv}}" { OBJECT_PATH "/xyz/openbmc_project/logging/entry/12535"; ARRAY "{sa{sv}}" { DICT_ENTRY "sa{sv}" { STRING "org.freedesktop.DBus.Peer"; ARRAY "{sv}" { }; }; DICT_ENTRY "sa{sv}" { STRING "org.freedesktop.DBus.Introspectable"; ARRAY "{sv}" { }; }; DICT_ENTRY "sa{sv}" { STRING "org.freedesktop.DBus.Properties"; ARRAY "{sv}" { }; }; DICT_ENTRY "sa{sv}" { STRING "com.ibm.Logging.Policy"; ARRAY "{sv}" { DICT_ENTRY "sv" { STRING "EventID"; VARIANT "s" { STRING "None"; }; }; DICT_ENTRY "sv" { STRING "Description"; VARIANT "s" { STRING "An internal BMC error occurred"

So I would expect the log message to say: Internal BMC error occurred. That would help the field on how to proceed with troubleshooting.

I did notice that customer has the 1.14 openbmctool version. Could not get them to update the tool to see if error symptom changed, they got upset and is now demanding a new system.

yesyjim avatar Feb 13 '20 20:02 yesyjim

@thalerj Can you see if this is covered in the new policy table?

yesyjim avatar Feb 13 '20 21:02 yesyjim

@yesyjim This case is not covered in the policy table as the expectation is that a sensor name or address would be provided here. @spinler How do you feel about rolling this into a generic event of an internal BMC error when we fail a policy table match?

thalerj avatar Feb 13 '20 21:02 thalerj

@yesyjim do you also have the output from the elogall.log file that mentions the Sensor.Device.Error.ReadFailure?

spinler avatar Feb 13 '20 21:02 spinler

@spinler Sure I have the openbmc data in ecurep for that customer. 58119,057,649. https://ecurep.mainz.de.ibm.com/ae5/#id=58119%2C057%2C649&path=58119%2C057%2C649%2Fmail20200212-000517-Joseph_Chen%2F

58119-elogall_log.txt

yesyjim avatar Feb 13 '20 22:02 yesyjim