encoderd crashed during onroad

[Hubblesphere]

Describe the bug

Normal drive, go to engage and get alert: "Process not running: encoderd"

reboot fixed issue, unsure about repeatability.

Provide a route where the issue occurs

8d8427a2116eace2|2022-11-02--08-40-28--0

openpilot version

b31b0310444e83dc8e151767f57ee21f46f1515b

Additional info

No response

[Hubblesphere]

A second instance of encoderd not running shortly after on road:

8d8427a2116eace2|2022-11-09--10-57-18--0

[adeebshihadeh]

Looks like there's two different encoderd crashes:

a few seconds after encoderd startup

[53964.495079] MAIN 0 kernel - msm_vidc: info: Opening video instance: 0000000000000000, 0
[53964.495129] MAIN 0 kernel - msm_vidc: info: Opening video instance: 0000000000000000, 0
[53964.495196] MAIN 0 kernel - msm_vidc: info: Opening video instance: 0000000000000000, 0
[53964.495380] MAIN 0 kernel - msm_vidc: info: Opening video instance: 0000000000000000, 0
[53967.530849] MAIN 0 kernel - msm_vidc:  err: State not recognized
[53967.530928] MAIN 0 kernel - msm_vidc:  err: Failed to move from state: 16 to 13
[53967.531024] MAIN 0 kernel - msm_vidc:  err: msm_comm_kill_session: no session to kill for inst 0000000000000000
[53967.531084] MAIN 0 kernel - msm_vidc:  err: Failed to move inst: 0000000000000000 to state 13
[53967.531148] MAIN 0 kernel - msm_vidc:  err: Failed STOP Streaming inst = 0000000000000000 on cap = 9
[53967.531221] MAIN 0 kernel - msm_vidc: info: Closed video instance: 0000000000000000
[53967.531282] MAIN 0 kernel - msm_vidc:  err: State not recognized
[53967.531342] MAIN 0 kernel - msm_vidc:  err: Failed to move from state: 16 to 13
[53967.531394] MAIN 0 kernel - msm_vidc:  err: msm_comm_kill_session: no session to kill for inst 0000000000000000
[53967.531449] MAIN 0 kernel - msm_vidc:  err: Failed to move inst: 0000000000000000 to state 13
[53967.531509] MAIN 0 kernel - msm_vidc:  err: Failed STOP Streaming inst = 0000000000000000 on cap = 9
[53967.531589] MAIN 0 kernel - msm_vidc: info: Closed video instance: 0000000000000000
[53967.531643] MAIN 0 kernel - msm_vidc:  err: State not recognized
[53967.531703] MAIN 0 kernel - msm_vidc:  err: Failed to move from state: 16 to 13
[53967.531760] MAIN 0 kernel - msm_vidc:  err: msm_comm_kill_session: no session to kill for inst 0000000000000000
[53967.531838] MAIN 0 kernel - msm_vidc:  err: Failed to move inst: 0000000000000000 to state 13
[53967.531890] MAIN 0 kernel - msm_vidc:  err: Failed STOP Streaming inst = 0000000000000000 on cap = 9
[53967.531948] MAIN 0 kernel - msm_vidc: info: Closed video instance: 0000000000000000
[53967.994982] MAIN 0 kernel - msm_vidc:  err: State not recognized
[53967.995078] MAIN 0 kernel - msm_vidc:  err: Failed to move from state: 16 to 13
[53967.995132] MAIN 0 kernel - msm_vidc:  err: msm_comm_kill_session: no session to kill for inst 0000000000000000
[53967.995220] MAIN 0 kernel - msm_vidc:  err: Failed to move inst: 0000000000000000 to state 13
[53967.995271] MAIN 0 kernel - msm_vidc:  err: Failed STOP Streaming inst = 0000000000000000 on cap = 9
[53967.995329] MAIN 0 kernel - msm_vidc: info: Closed video instance: 0000000000000000

In the middle of the drive

{'tombstone': 'selfdrive/loggerd/encoderd - Signal: 6 (SIGABRT) -  in V4LEncoder::dequeue_handler  at selfdrive/loggerd/encoder/v4l_encoder.cc:111'}

[kevinharbin]

Same issue on Release today, wouldn't go onroad because "Process not running: encoderd" 35c004509e3057f2|2023-03-16--06-42-13--0

[sshane]

Another occurrence: 4822a427b188122a|2023-08-14--16-22-21

[sshane]

Another ff2bd20623fcaeaa|2023-08-18--10-13-46

[adeebshihadeh]

Happened on one of the CI devices recently, and in that case, it was due to camera re-alignment and frame ID skips.

[YassineYousfi]

Just happened in CI again in 3799fe4.