botasaurus
botasaurus copied to clipboard
Main task never ends in case of there is no result in google map.
how to reproduce
Search for string "botototootototototototototootototototoot"
it will give following error :
Traceback (most recent call last):
File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 192, in run_task
self.update_parent_task(task_id, result)
File "C:\Python312\Lib\site-packages\botasaurus_server\retry_on_db_error.py", line 37, in wrapper
raise e
File "C:\Python312\Lib\site-packages\botasaurus_server\retry_on_db_error.py", line 26, in wrapper
created_result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\botasaurus_server\retry_on_db_error.py", line 77, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 205, in update_parent_task
self.complete_parent_task_if_possible(parent_id, Server.get_remove_duplicates_by(scraper_name), result)
File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 230, in complete_parent_task_if_possible
fn()
File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 227, in <lambda>
fn = lambda: TaskHelper.read_clean_save_task(parent_id, remove_duplicates_by, status)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\botasaurus_server\task_helper.py", line 190, in read_clean_save_task
rs = normalize_dicts_by_fieldnames(TaskResults.get_all_task(parent_id))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\botasaurus_server\cleaners.py", line 35, in normalize_dicts_by_fieldnames
for item in data:
TypeError: 'NoneType' object is not iterable
Error in run_task 'NoneType' object is not iterable
Potential Fix:
Return an empty list from function normalize_dicts_by_fieldnames
def normalize_dicts_by_fieldnames(data):
if data is None or len(data) == 0:
return []