python-mapswipe-workers
python-mapswipe-workers copied to clipboard
wrong task in results leads to ForeignKeyViolation
When there is a result such as the one below in Firebase, this will lead to an error when transfering the results to postgres. This is linked to this issue on the client side https://github.com/mapswipe/mapswipe/issues/619
The task t34299
belongs to another group and triggers the Foreign Key violation later on.
A very simple approach could look like this (it's more or less what I do manually):
- first try to just transfer the data without any check
- if the transfer fails --> get the information from the error log which group and task is affected
- remove the wrong task for the affected group
- try to run the transfer again
- repeat if it still fails
There are probably nicer ways to do this, e.g.
- import results to table
results_temp
(this table does not have any keys defined) - check the
results_temp
table for potential ForeignKeyViolation - remove the all faulty results from
results_temp
- insert
results_temp
intoresults
"endTime" : "2022-03-25T22:06:24.578Z",
"results" : {
"t34299" : 0,
"t35150" : 0,
"t35151" : 0,
"t35152" : 0,
"t35153" : 0,
"t35154" : 0,
"t35155" : 0,
"t35156" : 0,
"t35157" : 0,
"t35158" : 0,
"t35159" : 0,
"t35160" : 0,
"t35161" : 0,
"t35162" : 0,
"t35163" : 0,
"t35164" : 0,
"t35165" : 0,
"t35166" : 0,
"t35167" : 0,
"t35168" : 0,
"t35169" : 0,
"t35170" : 0,
"t35171" : 0,
"t35172" : 0,
"t35173" : 0,
"t35174" : 0,
"t35175" : 0,
"t35176" : 0,
"t35177" : 0,
"t35178" : 0,
"t35179" : 0,
"t35180" : 0,
"t35181" : 0,
"t35182" : 0,
"t35183" : 0,
"t35184" : 0,
"t35185" : 0,
"t35186" : 0
},
"startTime" : "2022-03-25T22:06:16.458Z"
}
DETAIL: Key (project_id, group_id, task_id)=(-Myiz41plx7m5ya_JlCf, g1050, t34299) is not present in table "tasks".
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/mapswipe_workers/firebase_to_postgres/transfer_results.py", line 93, in transfer_results_for_project
save_results_to_postgres(results_file)
File "/usr/local/lib/python3.8/dist-packages/mapswipe_workers/firebase_to_postgres/transfer_results.py", line 292, in save_results_to_postgres
p_con.query(query_insert_results)
File "/usr/local/lib/python3.8/dist-packages/mapswipe_workers/auth.py", line 64, in query
self._db_cur.execute(query, data)
psycopg2.errors.ForeignKeyViolation: insert or update on table "results" violates foreign key constraint "results_project_id_fkey2"
DETAIL: Key (project_id, group_id, task_id)=(-Myiz41plx7m5ya_JlCf, g1050, t34299) is not present in table "tasks".