Unpack workers land in indefinite crash loop

[maringuu]

FACT version

4.1.1

Environment

Ubuntu 22.04.4 LTS

Steps to reproduce

Upload the available firmware here. The exact image I downloaded, is the one that has size 6.16 MB.

Expeced Behavior

It should either work or tell me that is cannot unpack once. It should not cause infinite endless loops of error messages.it

Backend logs

fact_main_backend.log

[2024-02-19 09:09:24][db_interface_base][ERROR]: Database error when trying to write to the database: (psycopg2.errors.FeatureNotSupported) conversion between UTF8 and SQL_ASCII is not supported
LINE 1: ...721812, ARRAY['no data lost']::VARCHAR[], 'null', '{"plugin_...
                                                             ^

[SQL: INSERT INTO analysis (uid, plugin, plugin_version, system_version, analysis_date, summary, tags, result) VALUES (%(uid)s, %(plugin)s, %(plugin_version)s, %(system_version)s, %(analysis_date)s, %(summary)s::VARCHAR[], %(tags)s, %(result)s)]
[parameters: {'uid': 'b119ac55a25b0b435b42d20257528e0302397ba056e62e8961d353fe9b309b00_6495423', 'plugin': 'unpacker', 'plugin_version': '0.1', 'system_version': None, 'analysis_date': 1708333764.721812, 'summary': ['no data lost'], 'tags': 'null', 'result': '{"plugin_used": "SFX", "output": "\\n7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21\\np7zip Version 16.02 (locale=en_US.UTF-8,Ut ... (956 characters truncated) ...  "number_of_unpacked_files": 4, "number_of_unpacked_directories": 0, "entropy": 0.9909443139228516, "size_packed": 6494143, "size_unpacked": 8390260}'}]
(Background on this error at: https://sqlalche.me/e/14/tw8g)
Traceback (most recent call last):
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.FeatureNotSupported: conversion between UTF8 and SQL_ASCII is not supported
LINE 1: ...721812, ARRAY['no data lost']::VARCHAR[], 'null', '{"plugin_...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/opt/fact-v4.1.1/src/storage/db_interface_base.py", line 51, in get_read_write_session
    session.commit()
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1451, in commit
    self._transaction.commit(_to_root=self.future)
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 829, in commit
    self._prepare_impl()
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
    self.session.flush()
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3444, in flush
    self._flush(objects)
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3583, in _flush
    with util.safe_reraise():
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
    raise exception
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3544, in _flush
    flush_context.execute()
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
    rec.execute(self)
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
    util.preloaded.orm_persistence.save_obj(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
    _emit_insert_statements(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements
    c = connection._execute_20(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
    ret = self._execute_context(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
    raise exception
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/root/venvs/fact-v4.1.1/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.NotSupportedError: (psycopg2.errors.FeatureNotSupported) conversion between UTF8 and SQL_ASCII is not supported
LINE 1: ...721812, ARRAY['no data lost']::VARCHAR[], 'null', '{"plugin_...
                                                             ^

[SQL: INSERT INTO analysis (uid, plugin, plugin_version, system_version, analysis_date, summary, tags, result) VALUES (%(uid)s, %(plugin)s, %(plugin_version)s, %(system_version)s, %(analysis_date)s, %(summary)s::VARCHAR[], %(tags)s, %(result)s)]
[parameters: {'uid': 'b119ac55a25b0b435b42d20257528e0302397ba056e62e8961d353fe9b309b00_6495423', 'plugin': 'unpacker', 'plugin_version': '0.1', 'system_version': None, 'analysis_date': 1708333764.721812, 'summary': ['no data lost'], 'tags': 'null', 'result': '{"plugin_used": "SFX", "output": "\\n7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21\\np7zip Version 16.02 (locale=en_US.UTF-8,Ut ... (956 characters truncated) ...  "number_of_unpacked_files": 4, "number_of_unpacked_directories": 0, "entropy": 0.9909443139228516, "size_packed": 6494143, "size_unpacked": 8390260}'}]
(Background on this error at: https://sqlalche.me/e/14/tw8g)
[2024-02-19 09:09:24][analysis][ERROR]: Could not add b119ac55a25b0b435b42d20257528e0302397ba056e62e8961d353fe9b309b00_6495423 to the DB: Database error when trying to write to the database
[2024-02-19 09:09:25][db_interface_base][ERROR]: Database error when trying to write to the database: Trying to add object but no parent object was found in DB: ['b119ac55a25b0b435b42d20257528e0302397ba056e62e8961d353fe9b309b00_6495423']
Traceback (most recent call last):
  File "/root/opt/fact-v4.1.1/src/storage/db_interface_base.py", line 50, in get_read_write_session
    yield session
  File "/root/opt/fact-v4.1.1/src/storage/db_interface_backend.py", line 41, in insert_file_object
    self._update_parents(file_object.parent_firmware_uids, file_object.parents, fo_entry, session)
  File "/root/opt/fact-v4.1.1/src/storage/db_interface_backend.py", line 49, in _update_parents
    self._update_entries(session, fo_entry.parent_files, parent_uids, 'parent')
  File "/root/opt/fact-v4.1.1/src/storage/db_interface_backend.py", line 55, in _update_entries
    raise DbInterfaceError(f'Trying to add object but no {label} object was found in DB: {uid_list}')
storage.db_interface_base.DbInterfaceError: Trying to add object but no parent object was found in DB: ['b119ac55a25b0b435b42d20257528e0302397ba056e62e8961d353fe9b309b00_6495423']

[jstucke]

It seems I'm unable to reproduce this bug in the latest master version, so I guess it is fixed already

[ElDavoo]

Cannot reproduce