prefect-soda-core
prefect-soda-core copied to clipboard
Error when an error occurs in some soda check
When all tests pass, no error occurs Some libs used prefect==2.4.2 prefect_shell==0.1.3 prefect-soda-core[postgres] prefect-soda-core[athena] soda-core soda-core-postgres soda-core-athena MarkupSafe==2.0.1 PyAthena
[17:01:56] c20 nulos [PASSED]
[17:01:56] C20 deve ser maior ou igual a 0 [PASSED]
[17:01:56] C40 nulos [PASSED]
[17:01:56] C40 deve ser maior ou igual a 0 [PASSED]
[17:01:56] db_maritimo [Duplicated] in cabotagem
[17:01:56] CEs duplicados [PASSED]
[17:01:56] db_maritimo [Today] in cabotagem
[17:01:56] Sem registros criados na data de hoje [PASSED]
[17:01:56] Data Operação null [PASSED]
05:01:56 PM
soda_scan_execute-1578bf38-0
[17:01:56] 2/30 checks WARNED:
05:01:56 PM
soda_scan_execute-1578bf38-0
[17:01:56] db_maritimo [Five] in cabotagem
[17:01:56] tipo_fcl nulos [WARNED]
[17:01:56] check_value: 1.03
[17:01:56] row_count: 7749
[17:01:56] missing_count: 80
[17:01:56] db_maritimo [Duplicated] in cabotagem
[17:01:56] Embarque duplicados (id_porto_carga,id_porto_origem,nrblconhecimento,lista_fcl iguais) [WARNED]
[17:01:56] check_value: 2368
05:01:56 PM
soda_scan_execute-1578bf38-0
[17:01:56] Only 2 warnings. 0 failures. 0 errors. 28 pass.
[17:01:56] Sending results to Soda Cloud
05:01:56 PM
soda_scan_execute-1578bf38-0
[17:02:03] Soda Cloud Trace: 7899457579506349419
05:02:03 PM
soda_scan_execute-1578bf38-0
Saving scan results to 2023-06-30T20:00:37.181228+00:00--soda_scan_execute-1578bf38-0.json
05:02:03 PM
soda_scan_execute-1578bf38-0
Encountered exception during execution:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 1214, in orchestrate_task_run
result = await task.fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect_soda_core/tasks.py", line 142, in soda_scan_execute
raise e
File "/usr/local/lib/python3.8/site-packages/prefect_soda_core/tasks.py", line 134, in soda_scan_execute
soda_logs = await shell_run_command.fn(
File "/usr/local/lib/python3.8/site-packages/prefect_shell/commands.py", line 103, in shell_run_command
raise RuntimeError(msg)
RuntimeError: Command failed with exit code 1:
Saving scan results to 2023-06-30T20:00:37.181228+00:00--soda_scan_execute-1578bf38-0.json
05:02:08 PM
soda_scan_execute-1578bf38-0
Encountered exception during execution:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 596, in orchestrate_flow_run
result = await run_sync(flow_call)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 117, in run_sync_in_interruptible_worker_thread
tg.start_soon(
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 597, in __aexit__
raise exceptions[0]
File "/usr/local/lib/python3.8/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 97, in capture_worker_thread_and_result
result = __fn(*args, **kwargs)
File "flows/Governanca/soda/soda_cabotagem_flow.py", line 26, in soda_cabotagem_captura_flow
run_soda_scan('psql',
File "/tmp/tmpsv7sxzh_prefect/flows/lib_classes/libsoda.py", line 30, in run_soda_scan
return soda_scan_execute(
File "/usr/local/lib/python3.8/site-packages/prefect/tasks.py", line 295, in __call__
return enter_task_run_engine(
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 736, in enter_task_run_engine
return run_async_from_worker_thread(begin_run)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 137, in run_async_from_worker_thread
return anyio.from_thread.run(call)
File "/usr/local/lib/python3.8/site-packages/anyio/from_thread.py", line 47, in run
return asynclib.run_async_from_thread(func, *args)
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 906, in run_async_from_thread
return f.result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
raise self._exception
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 874, in get_task_call_return_value
return await future._result()
File "/usr/local/lib/python3.8/site-packages/prefect/futures.py", line 237, in _result
return final_state.result(raise_on_failure=raise_on_failure)
File "/usr/local/lib/python3.8/site-packages/prefect/orion/schemas/states.py", line 145, in result
raise data
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 1214, in orchestrate_task_run
result = await task.fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect_soda_core/tasks.py", line 142, in soda_scan_execute
raise e
File "/usr/local/lib/python3.8/site-packages/prefect_soda_core/tasks.py", line 134, in soda_scan_execute
soda_logs = await shell_run_command.fn(
File "/usr/local/lib/python3.8/site-packages/prefect_shell/commands.py", line 103, in shell_run_command
raise RuntimeError(msg)
RuntimeError: Command failed with exit code 1: