airflow
airflow copied to clipboard
Published
20 hours ago •
apache
apache
Received SIGTERM. Terminating subprocesses.
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.9.1
What happened?
Sometimes some dags are failing with:
[2024-12-16, 07:33:00 UTC] {base.py:290} INFO - OpenMetadata: Processed 0 records, updated 0 records, filtered 0 records, found 0 errors
[2024-12-16, 07:33:35 UTC] {local_task_job_runner.py:288} WARNING - Recorded pid 99982 does not match the current pid 96297
[2024-12-16, 07:33:35 UTC] {local_task_job_runner.py:222} ▲▲▲ Log group end
[2024-12-16, 07:33:35 UTC] {process_utils.py:132} INFO - Sending Signals.SIGTERM to group 96297. PIDs of all processes in the group: [96297]
[2024-12-16, 07:33:35 UTC] {process_utils.py:87} INFO - Sending the signal Signals.SIGTERM to group 96297
[2024-12-16, 07:33:35 UTC] {taskinstance.py:2611} ERROR - Received SIGTERM. Terminating subprocesses.
[2024-12-16, 07:33:35 UTC] {common.py:304} INFO - Sending failed status from callback...
[2024-12-16, 07:33:35 UTC] {server_mixin.py:74} INFO - OpenMetadata client running with Server version [1.5.12] and Client version [1.5.12.0]
[2024-12-16, 07:33:35 UTC] {common.py:310} INFO - Sending status to Ingestion Pipeline tessera-sql4.online_view.dbo.FL_Polk_Circuit_View.testSuite.tessera-sql4_online_view_dbo_FL_Polk_Circuit_View_Pipeline
[2024-12-16, 07:33:36 UTC] {taskinstance.py:2905} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 465, in _execute_task
result = _execute_callable(context=context, **execute_callable_kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 432, in _execute_callable
return execute_callable(context=context, **execute_callable_kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 400, in wrapper
return func(self, *args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 235, in execute
return_value = self.execute_callable()
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 252, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis/workflows/ingestion/test_suite.py", line 48, in test_suite_workflow
workflow.execute()
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/base.py", line 187, in execute
self.execute_internal()
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 131, in execute_internal
processed_record = step.run(processed_record)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/api/step.py", line 109, in run
result: Either = self._run(record)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/processor/test_case_runner.py", line 111, in _run
test_results = [
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/processor/test_case_runner.py", line 114, in <listcomp>
if (test_case_result := self._run_test_case(test_case, test_suite_runner))
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/processor/test_case_runner.py", line 295, in _run_test_case
test_result = test_suite_runner.run_and_handle(test_case)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/runner/core.py", line 37, in run_and_handle
test_result = self.test_runner_interface.run_test_case(
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/interface/test_suite_interface.py", line 124, in run_test_case
return validator.run_validation()
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/validations/table/sqlalchemy/TableColumnCompletenessToMeetThreshold.py", line 224, in run_validation
null_missing_result = self.run_query_results(
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/data_quality/validations/mixins/sqa_validator_mixin.py", line 75, in run_query_results
value = dict(runner.dispatch_query_select_first(metric_fn)) # type: ignore
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/utils/timeout.py", line 59, in inner
result = fn(*args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/profiler/processor/runner.py", line 147, in dispatch_query_select_first
return self.select_first_from_table(*entities, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/utils/timeout.py", line 59, in inner
result = fn(*args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/profiler/processor/handle_partition.py", line 126, in handle_and_execute
return func(_self, *args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/metadata/profiler/processor/runner.py", line 102, in select_first_from_table
return query.first()
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2824, in first
return self.limit(1)._iter().first()
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2916, in _iter
result = self.session.execute(
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1717, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
ret = self._execute_context(
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
self._handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2138, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
File "/home/airflow/.local/lib/python3.10/site-packages/pytds/cursor.py", line 322, in execute
self._session.execute(operation, params)
File "/home/airflow/.local/lib/python3.10/site-packages/pytds/tds_session.py", line 916, in execute
self.begin_response()
File "/home/airflow/.local/lib/python3.10/site-packages/pytds/tds_session.py", line 1261, in begin_response
return self._reader.begin_response()
File "/home/airflow/.local/lib/python3.10/site-packages/pytds/tds_reader.py", line 201, in begin_response
self._read_packet()
File "/home/airflow/.local/lib/python3.10/site-packages/pytds/tds_reader.py", line 214, in _read_packet
received = self._transport.recv_into(
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 2613, in signal_handler
raise AirflowTaskTerminated("Task received SIGTERM signal")
airflow.exceptions.AirflowTaskTerminated: Task received SIGTERM signal
[2024-12-16, 07:33:36 UTC] {common.py:304} INFO - Sending failed status from callback...
Then task restarts with
*** Found local files:
*** * /opt/airflow/logs/dag_id=tessera-sql4_online_view_dbo_FL_Polk_Circuit_View_Pipeline/run_id=manual__2024-12-16T07:30:55+00:00/task_id=test_suite_task/attempt=2.log
[2024-12-16, 07:33:30 UTC] {local_task_job_runner.py:120} ▶ Pre task execution logs
[2024-12-16, 07:33:31 UTC] {server_mixin.py:74} INFO - OpenMetadata client running with Server version [1.5.12] and Client version [1.5.12.0]
[2024-12-16, 07:33:33 UTC] {core.py:33} INFO - Executing test case TableColumnCompletenessToMeetThresholdTest for entity tessera-sql4.online_view.dbo.FL_Polk_Circuit_View
[2024-12-16, 07:33:33 UTC] {server_mixin.py:74} INFO - OpenMetadata client running with Server version [1.5.12] and Client version [1.5.12.0]
[2024-12-16, 07:33:33 UTC] {TableColumnCompletenessToMeetThreshold.py:204} INFO - TestParameters(min_threshold=15, max_threshold=15, columns_to_display=10)
[2024-12-16, 07:33:33 UTC] {TableColumnCompletenessToMeetThreshold.py:211} INFO - Columns to check:213:[Column('Source_ID', VARCHAR(), table=<FL_Polk_Circuit_View>, key='source_id', primary_key=True, nullable=False), Column('Offender_ID', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offender_id'), Column('File_Data_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='file_data_date'), Column('DC_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='dc_number'), Column('Inmate_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='inmate_number'), Column('FBI_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='fbi_number'), Column('State_ID_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='state_id_number'), Column('State_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='state_code'), Column('Booking_no', VARCHAR(), table=<FL_Polk_Circuit_View>, key='booking_no'), Column('Full_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='full_name'), Column('Last_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_name'), Column('First_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='first_name'), Column('Middle_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='middle_name'), Column('Suffix', VARCHAR(), table=<FL_Polk_Circuit_View>, key='suffix'), Column('DOB', VARCHAR(), table=<FL_Polk_Circuit_View>, key='dob'), Column('Age', VARCHAR(), table=<FL_Polk_Circuit_View>, key='age'), Column('Birth_County', VARCHAR(), table=<FL_Polk_Circuit_View>, key='birth_county'), Column('Birth_State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='birth_state'), Column('Birth_Country', VARCHAR(), table=<FL_Polk_Circuit_View>, key='birth_country'), Column('Gender_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='gender_code'), Column('Gender', VARCHAR(), table=<FL_Polk_Circuit_View>, key='gender'), Column('Race_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='race_code'), Column('Race', VARCHAR(), table=<FL_Polk_Circuit_View>, key='race'), Column('Ethnicity_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='ethnicity_code'), Column('Ethnicity', VARCHAR(), table=<FL_Polk_Circuit_View>, key='ethnicity'), Column('Eye_Color', VARCHAR(), table=<FL_Polk_Circuit_View>, key='eye_color'), Column('Hair_Color', VARCHAR(), table=<FL_Polk_Circuit_View>, key='hair_color'), Column('Skin_Color', VARCHAR(), table=<FL_Polk_Circuit_View>, key='skin_color'), Column('Height', VARCHAR(), table=<FL_Polk_Circuit_View>, key='height'), Column('Weight', VARCHAR(), table=<FL_Polk_Circuit_View>, key='weight'), Column('Physical_Build', VARCHAR(), table=<FL_Polk_Circuit_View>, key='physical_build'), Column('SSN', VARCHAR(), table=<FL_Polk_Circuit_View>, key='ssn'), Column('Driver_License_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='driver_license_number'), Column('Driver_License_State_Issue', VARCHAR(), table=<FL_Polk_Circuit_View>, key='driver_license_state_issue'), Column('Other_DOB_Used', VARCHAR(), table=<FL_Polk_Circuit_View>, key='other_dob_used'), Column('Citizenship', VARCHAR(), table=<FL_Polk_Circuit_View>, key='citizenship'), Column('Military_Service', VARCHAR(), table=<FL_Polk_Circuit_View>, key='military_service'), Column('Military_Branch', VARCHAR(), table=<FL_Polk_Circuit_View>, key='military_branch'), Column('Military_Discharge', VARCHAR(), table=<FL_Polk_Circuit_View>, key='military_discharge'), Column('Military_Discharge_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='military_discharge_date'), Column('ScareMarkTatoo', VARCHAR(), table=<FL_Polk_Circuit_View>, key='scaremarktatoo'), Column('Photo', VARCHAR(), table=<FL_Polk_Circuit_View>, key='photo'), Column('Vehicle_year', VARCHAR(), table=<FL_Polk_Circuit_View>, key='vehicle_year'), Column('Vehicle_color', VARCHAR(), table=<FL_Polk_Circuit_View>, key='vehicle_color'), Column('Vehicle_mark', VARCHAR(), table=<FL_Polk_Circuit_View>, key='vehicle_mark'), Column('Vehicle_model', VARCHAR(), table=<FL_Polk_Circuit_View>, key='vehicle_model'), Column('Alias_full_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='alias_full_name'), Column('Alias_First_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='alias_first_name'), Column('Alias_Middle_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='alias_middle_name'), Column('Alias_Last_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='alias_last_name'), Column('Alias_Suffix', VARCHAR(), table=<FL_Polk_Circuit_View>, key='alias_suffix'), Column('Judge', VARCHAR(), table=<FL_Polk_Circuit_View>, key='judge'), Column('Attorney', VARCHAR(), table=<FL_Polk_Circuit_View>, key='attorney'), Column('Party_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_code'), Column('Party_Relationship', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_relationship'), Column('Party_full_name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_full_name'), Column('Party_Last_name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_last_name'), Column('Party_First_name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_first_name'), Column('Party_Middle_name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_middle_name'), Column('Party_Suffix_name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_suffix_name'), Column('Party_DOB', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_dob'), Column('Party_Age', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_age'), Column('Party_Sex', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_sex'), Column('Party_Address', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_address'), Column('Party_City', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_city'), Column('Party_State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_state'), Column('Party_Zip', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_zip'), Column('Party_Phone', VARCHAR(), table=<FL_Polk_Circuit_View>, key='party_phone'), Column('Full_address', VARCHAR(), table=<FL_Polk_Circuit_View>, key='full_address'), Column('Address_1', VARCHAR(), table=<FL_Polk_Circuit_View>, key='address_1'), Column('Address_2', VARCHAR(), table=<FL_Polk_Circuit_View>, key='address_2'), Column('Address_3', VARCHAR(), table=<FL_Polk_Circuit_View>, key='address_3'), Column('City', VARCHAR(), table=<FL_Polk_Circuit_View>, key='city'), Column('County', VARCHAR(), table=<FL_Polk_Circuit_View>, key='county'), Column('State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='state'), Column('Zip', VARCHAR(), table=<FL_Polk_Circuit_View>, key='zip'), Column('Phone', VARCHAR(), table=<FL_Polk_Circuit_View>, key='phone'), Column('Last_Institution_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_institution_code'), Column('Last_Institution', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_institution'), Column('Last_Institution_Address', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_institution_address'), Column('Last_Institution_City', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_institution_city'), Column('Last_Institution_State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_institution_state'), Column('Last_Institution_Zip', VARCHAR(), table=<FL_Polk_Circuit_View>, key='last_institution_zip'), Column('Institution_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_code'), Column('Institution_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_name'), Column('Institution_Address', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_address'), Column('Institution_City', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_city'), Column('Institution_State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_state'), Column('Institution_Zip', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_zip'), Column('Institution_Phone', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_phone'), Column('Institution_Details', VARCHAR(), table=<FL_Polk_Circuit_View>, key='institution_details'), Column('Released_to_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='released_to_name'), Column('Released_to_Address', VARCHAR(), table=<FL_Polk_Circuit_View>, key='released_to_address'), Column('Released_to_City', VARCHAR(), table=<FL_Polk_Circuit_View>, key='released_to_city'), Column('Released_to_State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='released_to_state'), Column('Released_to_Zip', VARCHAR(), table=<FL_Polk_Circuit_View>, key='released_to_zip'), Column('Release_Information', VARCHAR(), table=<FL_Polk_Circuit_View>, key='release_information'), Column('Court_Case_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_case_number'), Column('Court_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_code'), Column('Court_Name', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_name'), Column('Court_County', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_county'), Column('Court_type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_type'), Column('Case_Category_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_category_code'), Column('Case_Category', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_category'), Column('Case_Year', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_year'), Column('Case_Type_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_type_code'), Column('Case_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_type'), Column('Arresting_Agency', VARCHAR(), table=<FL_Polk_Circuit_View>, key='arresting_agency'), Column('Filing_Agency', VARCHAR(), table=<FL_Polk_Circuit_View>, key='filing_agency'), Column('Date_Filed', VARCHAR(), table=<FL_Polk_Circuit_View>, key='date_filed'), Column('Arrest_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='arrest_date'), Column('Arraignment_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='arraignment_date'), Column('Court_Clerk_Phone', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_clerk_phone'), Column('Fine', VARCHAR(), table=<FL_Polk_Circuit_View>, key='fine'), Column('Court_cost', VARCHAR(), table=<FL_Polk_Circuit_View>, key='court_cost'), Column('Case_disposition_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_disposition_code'), Column('Case_disposition', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_disposition'), Column('Rebuttal', VARCHAR(), table=<FL_Polk_Circuit_View>, key='rebuttal'), Column('Bail_amount', VARCHAR(), table=<FL_Polk_Circuit_View>, key='bail_amount'), Column('Bond', VARCHAR(), table=<FL_Polk_Circuit_View>, key='bond'), Column('Offender_Status_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offender_status_code'), Column('Offender_Status', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offender_status'), Column('Case_status', VARCHAR(), table=<FL_Polk_Circuit_View>, key='case_status'), Column('NCIC_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='ncic_code'), Column('Offense_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_code'), Column('Offense_Prefix', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_prefix'), Column('Offense', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense'), Column('Offense_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_type'), Column('Offense_Class', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_class'), Column('Offense_Degree', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_degree'), Column('Offense_Level', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_level'), Column('Amended_Offense', VARCHAR(), table=<FL_Polk_Circuit_View>, key='amended_offense'), Column('Amended_Offense_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='amended_offense_type'), Column('Amended_Offense_class', VARCHAR(), table=<FL_Polk_Circuit_View>, key='amended_offense_class'), Column('Amended_Offense_Degree', VARCHAR(), table=<FL_Polk_Circuit_View>, key='amended_offense_degree'), Column('Amended_Offense_Level', VARCHAR(), table=<FL_Polk_Circuit_View>, key='amended_offense_level'), Column('Sub_Offense_Code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sub_offense_code'), Column('Sub_Offense_Prefix', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sub_offense_prefix'), Column('Sub_Offense', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sub_offense'), Column('Sub_Offense_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sub_offense_type'), Column('Sub_Offense_Class', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sub_offense_class'), Column('Charge_No', VARCHAR(), table=<FL_Polk_Circuit_View>, key='charge_no'), Column('Total_Charge_counts', VARCHAR(), table=<FL_Polk_Circuit_View>, key='total_charge_counts'), Column('Offense_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_date'), Column('Citation_Number', VARCHAR(), table=<FL_Polk_Circuit_View>, key='citation_number'), Column('Offense_City', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_city'), Column('Offense_County', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_county'), Column('Offense_State', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_state'), Column('Original_Plea_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='original_plea_code'), Column('Original_Plea', VARCHAR(), table=<FL_Polk_Circuit_View>, key='original_plea'), Column('Original_Plea_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='original_plea_date'), Column('Plea_Withdrawn_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='plea_withdrawn_date'), Column('New_Plea_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='new_plea_code'), Column('New_Plea', VARCHAR(), table=<FL_Polk_Circuit_View>, key='new_plea'), Column('Verdict_Finding_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='verdict_finding_code'), Column('Verdict_Finding', VARCHAR(), table=<FL_Polk_Circuit_View>, key='verdict_finding'), Column('Verdict_Conviction_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='verdict_conviction_date'), Column('County_Convicted', VARCHAR(), table=<FL_Polk_Circuit_View>, key='county_convicted'), Column('State_Convicted', VARCHAR(), table=<FL_Polk_Circuit_View>, key='state_convicted'), Column('Offense_Disposition_code', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_disposition_code'), Column('Offense_Disposition', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_disposition'), Column('Offense_Disposition_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='offense_disposition_date'), Column('Warrant_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='warrant_date'), Column('Sentence_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_type'), Column('Sentence_Max_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_max_years'), Column('Sentence_Max_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_max_months'), Column('Sentence_Max_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_max_days'), Column('Sentence_Min_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_min_years'), Column('Sentence_Min_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_min_months'), Column('Sentence_Min_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_min_days'), Column('Sentence_length', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_length'), Column('Sentence_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_date'), Column('Sentence_Begin_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_begin_date'), Column('Sentence_Details', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_details'), Column('Parole_Start_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_start_date'), Column('Parole_End_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_end_date'), Column('Parole_Max_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_max_years'), Column('Parole_Max_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_max_months'), Column('Parole_Max_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_max_days'), Column('Parole_Min_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_min_years'), Column('Parole_Min_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_min_months'), Column('Parole_Min_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='parole_min_days'), Column('Probation_Start_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_start_date'), Column('Probation_End_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_end_date'), Column('Probation_Max_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_max_years'), Column('Probation_Max_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_max_months'), Column('Probation_Max_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_max_days'), Column('Probation_Min_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_min_years'), Column('Probation_Min_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_min_months'), Column('Probation_Min_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_min_days'), Column('Probation_Agency', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_agency'), Column('Probation_length', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_length'), Column('Maximum_Release_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='maximum_release_date'), Column('Scheduled_Release_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='scheduled_release_date'), Column('Actual_Release_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='actual_release_date'), Column('Release_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='release_type'), Column('Time_Served_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='time_served_years'), Column('Time_Served_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='time_served_months'), Column('Time_Served_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='time_served_days'), Column('Time_Served_length', VARCHAR(), table=<FL_Polk_Circuit_View>, key='time_served_length'), Column('Probation_Follows_Y_N', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_follows_y_n'), Column('Supervision_County', VARCHAR(), table=<FL_Polk_Circuit_View>, key='supervision_county'), Column('Sentence_Status', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_status'), Column('Probation_Parole_Status', VARCHAR(), table=<FL_Polk_Circuit_View>, key='probation_parole_status'), Column('Sentence_Comments1', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_comments1'), Column('Sentence_Comments2', VARCHAR(), table=<FL_Polk_Circuit_View>, key='sentence_comments2'), Column('Suspended_Jail_Years', VARCHAR(), table=<FL_Polk_Circuit_View>, key='suspended_jail_years'), Column('Suspended_Jail_Months', VARCHAR(), table=<FL_Polk_Circuit_View>, key='suspended_jail_months'), Column('Suspended_Jail_Days', VARCHAR(), table=<FL_Polk_Circuit_View>, key='suspended_jail_days'), Column('Admission_Date', VARCHAR(), table=<FL_Polk_Circuit_View>, key='admission_date'), Column('Admission_Type', VARCHAR(), table=<FL_Polk_Circuit_View>, key='admission_type'), Column('EPR_File_Record_Date', DATETIME(), table=<FL_Polk_Circuit_View>, key='epr_file_record_date'), Column('address_verification_date', VARCHAR(), table=<FL_Polk_Circuit_View>)]
[2024-12-16, 07:33:33 UTC] {TableColumnCompletenessToMeetThreshold.py:55} INFO - Getting previous test results
[2024-12-16, 07:33:33 UTC] {TableColumnCompletenessToMeetThreshold.py:324} INFO - test_result.testCaseStatus: Success
[2024-12-16, 07:33:33 UTC] {TableColumnCompletenessToMeetThreshold.py:336} INFO - Found previous test results: {"Source_ID": 100.0, "Offender_ID": 100.0, "File_Data_Date": 0.0, "DC_Number": 96.58, "Inmate_Number": 0.0, "FBI_Number": 0.0, "State_ID_Number": 96.58, "State_Code": 0.0, "Booking_no": 0.0, "Full_Name": 100.0, "Last_Name": 100.0, "First_Name": 100.0, "Middle_Name": 84.98, "Suffix": 0.0, "DOB": 99.84, "Age": 0.0, "Birth_County": 0.0, "Birth_State": 0.0, "Birth_Country": 0.0, "Gender_Code": 0.0, "Gender": 0.0, "Race_Code": 0.0, "Race": 0.0, "Ethnicity_Code": 0.0, "Ethnicity": 0.0, "Eye_Color": 0.0, "Hair_Color": 0.0, "Skin_Color": 0.0, "Height": 0.0, "Weight": 0.0, "Physical_Build": 0.0, "SSN": 0.0, "Driver_License_Number": 0.0, "Driver_License_State_Issue": 0.0, "Other_DOB_Used": 0.0, "Citizenship": 0.0, "Military_Service": 0.0, "Military_Branch": 0.0, "Military_Discharge": 0.0, "Military_Discharge_Date": 0.0, "ScareMarkTatoo": 0.0, "Photo": 0.0, "Vehicle_year": 0.0, "Vehicle_color": 0.0, "Vehicle_mark": 0.0, "Vehicle_model": 0.0, "Alias_full_Name": 0.0, "Alias_First_Name": 0.0, "Alias_Middle_Name": 0.0, "Alias_Last_Name": 0.0, "Alias_Suffix": 0.0, "Judge": 99.34, "Attorney": 0.0, "Party_code": 0.0, "Party_Relationship": 0.0, "Party_full_name": 0.0, "Party_Last_name": 0.0, "Party_First_name": 0.0, "Party_Middle_name": 0.0, "Party_Suffix_name": 0.0, "Party_DOB": 0.0, "Party_Age": 0.0, "Party_Sex": 0.0, "Party_Address": 0.0, "Party_City": 0.0, "Party_State": 0.0, "Party_Zip": 0.0, "Party_Phone": 0.0, "Full_address": 100.0, "Address_1": 99.55, "Address_2": 5.47, "Address_3": 0.0, "City": 99.41, "County": 0.0, "State": 99.4, "Zip": 98.85, "Phone": 0.0, "Last_Institution_Code": 0.0, "Last_Institution": 0.0, "Last_Institution_Address": 0.0, "Last_Institution_City": 0.0, "Last_Institution_State": 0.0, "Last_Institution_Zip": 0.0, "Institution_Code": 0.0, "Institution_Name": 0.0, "Institution_Address": 0.0, "Institution_City": 0.0, "Institution_State": 0.0, "Institution_Zip": 0.0, "Institution_Phone": 0.0, "Institution_Details": 0.0, "Released_to_Name": 0.0, "Released_to_Address": 0.0, "Released_to_City": 0.0, "Released_to_State": 0.0, "Released_to_Zip": 0.0, "Release_Information": 0.0, "Court_Case_Number": 100.0, "Court_Code": 0.0, "Court_Name": 0.0, "Court_County": 0.0, "Court_type": 0.0, "Case_Category_Code": 0.0, "Case_Category": 0.0, "Case_Year": 0.0, "Case_Type_code": 0.0, "Case_Type": 0.0, "Arresting_Agency": 99.28, "Filing_Agency": 0.0, "Date_Filed": 99.29, "Arrest_Date": 97.27, "Arraignment_Date": 0.0, "Court_Clerk_Phone": 0.0, "Fine": 0.0, "Court_cost": 0.0, "Case_disposition_code": 0.0, "Case_disposition": 0.0, "Rebuttal": 0.0, "Bail_amount": 0.0, "Bond": 0.0, "Offender_Status_Code": 0.0, "Offender_Status": 0.0, "Case_status": 100.0, "NCIC_Code": 0.0, "Offense_Code": 97.78, "Offense_Prefix": 0.0, "Offense": 99.34, "Offense_Type": 99.27, "Offense_Class": 0.0, "Offense_Degree": 99.27, "Offense_Level": 0.0, "Amended_Offense": 0.0, "Amended_Offense_Type": 0.0, "Amended_Offense_class": 0.0, "Amended_Offense_Degree": 0.0, "Amended_Offense_Level": 0.0, "Sub_Offense_Code": 0.0, "Sub_Offense_Prefix": 0.0, "Sub_Offense": 0.0, "Sub_Offense_Type": 0.0, "Sub_Offense_Class": 0.0, "Charge_No": 99.34, "Total_Charge_counts": 0.0, "Offense_Date": 99.33, "Citation_Number": 4.28, "Offense_City": 0.0, "Offense_County": 0.0, "Offense_State": 0.0, "Original_Plea_code": 0.0, "Original_Plea": 0.0, "Original_Plea_Date": 0.0, "Plea_Withdrawn_Date": 0.0, "New_Plea_code": 0.0, "New_Plea": 0.0, "Verdict_Finding_code": 0.0, "Verdict_Finding": 0.0, "Verdict_Conviction_Date": 0.0, "County_Convicted": 0.0, "State_Convicted": 0.0, "Offense_Disposition_code": 94.15, "Offense_Disposition": 94.14, "Offense_Disposition_Date": 94.15, "Warrant_Date": 0.0, "Sentence_Type": 33.95, "Sentence_Max_Years": 0.0, "Sentence_Max_Months": 0.0, "Sentence_Max_Days": 0.0, "Sentence_Min_Years": 0.0, "Sentence_Min_Months": 0.0, "Sentence_Min_Days": 0.0, "Sentence_length": 33.98, "Sentence_Date": 0.0, "Sentence_Begin_Date": 0.0, "Sentence_Details": 26.35, "Parole_Start_Date": 0.0, "Parole_End_Date": 0.0, "Parole_Max_Years": 0.0, "Parole_Max_Months": 0.0, "Parole_Max_Days": 0.0, "Parole_Min_Years": 0.0, "Parole_Min_Months": 0.0, "Parole_Min_Days": 0.0, "Probation_Start_Date": 0.0, "Probation_End_Date": 0.0, "Probation_Max_Years": 0.0, "Probation_Max_Months": 0.0, "Probation_Max_Days": 0.0, "Probation_Min_Years": 0.0, "Probation_Min_Months": 0.0, "Probation_Min_Days": 0.0, "Probation_Agency": 0.0, "Probation_length": 13.61, "Maximum_Release_Date": 0.0, "Scheduled_Release_Date": 0.0, "Actual_Release_Date": 0.0, "Release_Type": 0.0, "Time_Served_Years": 0.0, "Time_Served_Months": 0.0, "Time_Served_Days": 0.0, "Time_Served_length": 0.0, "Probation_Follows_Y_N": 0.0, "Supervision_County": 0.0, "Sentence_Status": 0.0, "Probation_Parole_Status": 0.0, "Sentence_Comments1": 13.61, "Sentence_Comments2": 0.0, "Suspended_Jail_Years": 0.0, "Suspended_Jail_Months": 0.0, "Suspended_Jail_Days": 0.0, "Admission_Date": 0.0, "Admission_Type": 0.0, "EPR_File_Record_Date": 100.0, "address_verification_date": 0.0}
[2024-12-16, 07:33:33 UTC] {TableColumnCompletenessToMeetThreshold.py:346} INFO - Parsed JSON previous results: {'Source_ID': 100.0, 'Offender_ID': 100.0, 'File_Data_Date': 0.0, 'DC_Number': 96.58, 'Inmate_Number': 0.0, 'FBI_Number': 0.0, 'State_ID_Number': 96.58, 'State_Code': 0.0, 'Booking_no': 0.0, 'Full_Name': 100.0, 'Last_Name': 100.0, 'First_Name': 100.0, 'Middle_Name': 84.98, 'Suffix': 0.0, 'DOB': 99.84, 'Age': 0.0, 'Birth_County': 0.0, 'Birth_State': 0.0, 'Birth_Country': 0.0, 'Gender_Code': 0.0, 'Gender': 0.0, 'Race_Code': 0.0, 'Race': 0.0, 'Ethnicity_Code': 0.0, 'Ethnicity': 0.0, 'Eye_Color': 0.0, 'Hair_Color': 0.0, 'Skin_Color': 0.0, 'Height': 0.0, 'Weight': 0.0, 'Physical_Build': 0.0, 'SSN': 0.0, 'Driver_License_Number': 0.0, 'Driver_License_State_Issue': 0.0, 'Other_DOB_Used': 0.0, 'Citizenship': 0.0, 'Military_Service': 0.0, 'Military_Branch': 0.0, 'Military_Discharge': 0.0, 'Military_Discharge_Date': 0.0, 'ScareMarkTatoo': 0.0, 'Photo': 0.0, 'Vehicle_year': 0.0, 'Vehicle_color': 0.0, 'Vehicle_mark': 0.0, 'Vehicle_model': 0.0, 'Alias_full_Name': 0.0, 'Alias_First_Name': 0.0, 'Alias_Middle_Name': 0.0, 'Alias_Last_Name': 0.0, 'Alias_Suffix': 0.0, 'Judge': 99.34, 'Attorney': 0.0, 'Party_code': 0.0, 'Party_Relationship': 0.0, 'Party_full_name': 0.0, 'Party_Last_name': 0.0, 'Party_First_name': 0.0, 'Party_Middle_name': 0.0, 'Party_Suffix_name': 0.0, 'Party_DOB': 0.0, 'Party_Age': 0.0, 'Party_Sex': 0.0, 'Party_Address': 0.0, 'Party_City': 0.0, 'Party_State': 0.0, 'Party_Zip': 0.0, 'Party_Phone': 0.0, 'Full_address': 100.0, 'Address_1': 99.55, 'Address_2': 5.47, 'Address_3': 0.0, 'City': 99.41, 'County': 0.0, 'State': 99.4, 'Zip': 98.85, 'Phone': 0.0, 'Last_Institution_Code': 0.0, 'Last_Institution': 0.0, 'Last_Institution_Address': 0.0, 'Last_Institution_City': 0.0, 'Last_Institution_State': 0.0, 'Last_Institution_Zip': 0.0, 'Institution_Code': 0.0, 'Institution_Name': 0.0, 'Institution_Address': 0.0, 'Institution_City': 0.0, 'Institution_State': 0.0, 'Institution_Zip': 0.0, 'Institution_Phone': 0.0, 'Institution_Details': 0.0, 'Released_to_Name': 0.0, 'Released_to_Address': 0.0, 'Released_to_City': 0.0, 'Released_to_State': 0.0, 'Released_to_Zip': 0.0, 'Release_Information': 0.0, 'Court_Case_Number': 100.0, 'Court_Code': 0.0, 'Court_Name': 0.0, 'Court_County': 0.0, 'Court_type': 0.0, 'Case_Category_Code': 0.0, 'Case_Category': 0.0, 'Case_Year': 0.0, 'Case_Type_code': 0.0, 'Case_Type': 0.0, 'Arresting_Agency': 99.28, 'Filing_Agency': 0.0, 'Date_Filed': 99.29, 'Arrest_Date': 97.27, 'Arraignment_Date': 0.0, 'Court_Clerk_Phone': 0.0, 'Fine': 0.0, 'Court_cost': 0.0, 'Case_disposition_code': 0.0, 'Case_disposition': 0.0, 'Rebuttal': 0.0, 'Bail_amount': 0.0, 'Bond': 0.0, 'Offender_Status_Code': 0.0, 'Offender_Status': 0.0, 'Case_status': 100.0, 'NCIC_Code': 0.0, 'Offense_Code': 97.78, 'Offense_Prefix': 0.0, 'Offense': 99.34, 'Offense_Type': 99.27, 'Offense_Class': 0.0, 'Offense_Degree': 99.27, 'Offense_Level': 0.0, 'Amended_Offense': 0.0, 'Amended_Offense_Type': 0.0, 'Amended_Offense_class': 0.0, 'Amended_Offense_Degree': 0.0, 'Amended_Offense_Level': 0.0, 'Sub_Offense_Code': 0.0, 'Sub_Offense_Prefix': 0.0, 'Sub_Offense': 0.0, 'Sub_Offense_Type': 0.0, 'Sub_Offense_Class': 0.0, 'Charge_No': 99.34, 'Total_Charge_counts': 0.0, 'Offense_Date': 99.33, 'Citation_Number': 4.28, 'Offense_City': 0.0, 'Offense_County': 0.0, 'Offense_State': 0.0, 'Original_Plea_code': 0.0, 'Original_Plea': 0.0, 'Original_Plea_Date': 0.0, 'Plea_Withdrawn_Date': 0.0, 'New_Plea_code': 0.0, 'New_Plea': 0.0, 'Verdict_Finding_code': 0.0, 'Verdict_Finding': 0.0, 'Verdict_Conviction_Date': 0.0, 'County_Convicted': 0.0, 'State_Convicted': 0.0, 'Offense_Disposition_code': 94.15, 'Offense_Disposition': 94.14, 'Offense_Disposition_Date': 94.15, 'Warrant_Date': 0.0, 'Sentence_Type': 33.95, 'Sentence_Max_Years': 0.0, 'Sentence_Max_Months': 0.0, 'Sentence_Max_Days': 0.0, 'Sentence_Min_Years': 0.0, 'Sentence_Min_Months': 0.0, 'Sentence_Min_Days': 0.0, 'Sentence_length': 33.98, 'Sentence_Date': 0.0, 'Sentence_Begin_Date': 0.0, 'Sentence_Details': 26.35, 'Parole_Start_Date': 0.0, 'Parole_End_Date': 0.0, 'Parole_Max_Years': 0.0, 'Parole_Max_Months': 0.0, 'Parole_Max_Days': 0.0, 'Parole_Min_Years': 0.0, 'Parole_Min_Months': 0.0, 'Parole_Min_Days': 0.0, 'Probation_Start_Date': 0.0, 'Probation_End_Date': 0.0, 'Probation_Max_Years': 0.0, 'Probation_Max_Months': 0.0, 'Probation_Max_Days': 0.0, 'Probation_Min_Years': 0.0, 'Probation_Min_Months': 0.0, 'Probation_Min_Days': 0.0, 'Probation_Agency': 0.0, 'Probation_length': 13.61, 'Maximum_Release_Date': 0.0, 'Scheduled_Release_Date': 0.0, 'Actual_Release_Date': 0.0, 'Release_Type': 0.0, 'Time_Served_Years': 0.0, 'Time_Served_Months': 0.0, 'Time_Served_Days': 0.0, 'Time_Served_length': 0.0, 'Probation_Follows_Y_N': 0.0, 'Supervision_County': 0.0, 'Sentence_Status': 0.0, 'Probation_Parole_Status': 0.0, 'Sentence_Comments1': 13.61, 'Sentence_Comments2': 0.0, 'Suspended_Jail_Years': 0.0, 'Suspended_Jail_Months': 0.0, 'Suspended_Jail_Days': 0.0, 'Admission_Date': 0.0, 'Admission_Type': 0.0, 'EPR_File_Record_Date': 100.0, 'address_verification_date': 0.0}
[2024-12-16, 07:33:36 UTC] {TableColumnCompletenessToMeetThreshold.py:219} INFO - Row count:596759
[2024-12-16, 07:33:46 UTC] {local_task_job_runner.py:310} WARNING - State of this instance has been externally set to failed. Terminating instance.
[2024-12-16, 07:33:46 UTC] {local_task_job_runner.py:222} ▲▲▲ Log group end
[2024-12-16, 07:33:46 UTC] {process_utils.py:132} INFO - Sending Signals.SIGTERM to group 99982. PIDs of all processes in the group: [99982]
[2024-12-16, 07:33:46 UTC] {process_utils.py:87} INFO - Sending the signal Signals.SIGTERM to group 99982
[2024-12-16, 07:33:46 UTC] {taskinstance.py:2611} ERROR - Received SIGTERM. Terminating subprocesses.
[2024-12-16, 07:33:46 UTC] {common.py:304} INFO - Sending failed status from callback...
[2024-12-16, 07:33:46 UTC] {server_mixin.py:74} INFO - OpenMetadata client running with Server version [1.5.12] and Client version [1.5.12.0]
[2024-12-16, 07:33:46 UTC] {common.py:310} INFO - Sending status to Ingestion Pipeline tessera-sql4.online_view.dbo.FL_Polk_Circuit_View.testSuite.tessera-sql4_online_view_dbo_FL_Polk_Circuit_View_Pipeline
[2024-12-16, 07:33:47 UTC] {process_utils.py:80} INFO - Process psutil.Process(pid=99982, status='terminated', exitcode=0, started='07:33:30') (99982) terminated with exit code 0
What you think should happen instead?
DAG run is successful
How to reproduce
Create a couple of thousands DAGs. It's not a performance issue, CPU and workers are okay.
Operating System
linux (ec2, docker)
Versions of Apache Airflow Providers
No response
Deployment
Docker-Compose
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
![boring-cyborg[bot] avatar](https://avatars.githubusercontent.com/in/50386?v=4 "Published by a pub.dev verified publisher"){kind=small}
@serhiikovalenkoextcheckrcom Unfortunately I think it's going to be almost impossible for anyone to give you any specific answers to the problem you are facing but the possible causes of a SIGTERM in this context could be one of the following.
Scheduler Timeout
- The Airflow scheduler may have enforced a task timeout (e.g., due to exceeding the execution_timeout parameter) and sent SIGTERM to stop the task.
Manual Termination
- A user or system admin may have manually terminated the task using airflow tasks kill or similar commands.
Orphaned Process
- The PID mismatch suggests that the task may have been restarted or orphaned, causing Airflow to send SIGTERM to clean up inconsistent state.
Resource Constraints
- The system or container running Airflow might have terminated the process due to resource exhaustion (e.g., memory, CPU limits).
Upstream Dependency or DAG Conflict
- If the task was part of a larger workflow, another task or operator failure could trigger an early termination.
