Error handling in case of broken links in the DB

[ghost]

trafficstars

• [05/Oct/2019 17:10:42] "GET /api/translate?strings=[%22Birth%20Date%22,%22Death%20Date%22,%22Relationships%22,%22People%22,%22Families%22,%22Dashboard%22,%22Given%20name%22,%22Surname%22,%22Marriage%20Date%22,%22Name%22,%22Father%22,%22Mother%22,%22Married%22,%22Event%22,%22Place%22,%22Description%22,%22Date%22,%22Type%22,%22Events%22,%22Parents%22,%22Siblings%22,%22Children%22,%22Home%20Page%22,%22Details%22,%22in%22,%22Spouses%22,%22Family%20Tree%22,%22Database%20overview%22,%22Number%20of%20individuals%22,%22Number%20of%20families%22,%22Number%20of%20events%22,%22Number%20of%20places%22,%22Places%22,%22Type%22,%22and%22,%22Primary%22,%22Family%22,%22Gallery%22,%22Map%22,%22Unknown%22,%22Custom%22,%22Country%22,%22State%22,%22County%22,%22City%22,%22Parish%22,%22Locality%22,%22Street%22,%22Province%22,%22Region%22,%22Department%22,%22Neighborhood%22,%22District%22,%22Borough%22,%22Municipality%22,%22Town%22,%22Village%22,%22Hamlet%22,%22Farm%22,%22Building%22,%22Number%22,%22Number%20of%20generations:%22,%22Zoom%22,%22Author%22,%22Publication%20info%22,%22Source%22,%22Sources%22,%22Citation%22,%22Citations%22,%22Repository%22,%22Repositories%22,%22Note%22,%22Notes%22,%22Media%22] HTTP/1.1" 200 - INFO:werkzeug:127.0.0.1 - - [05/Oct/2019 17:14:46] "GET /api/tree HTTP/1.1" 500 - ERROR:werkzeug:Error on request: Traceback (most recent call last): File "/home/jeff/.local/lib/python3.7/site-packages/werkzeug/serving.py", line 304, in run_wsgi execute(self.server.app) File "/home/jeff/.local/lib/python3.7/site-packages/werkzeug/serving.py", line 292, in execute application_iter = app(environ, start_response) File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 2463, in call return self.wsgi_app(environ, start_response) File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 2449, in wsgi_app response = self.handle_exception(e) File "/home/jeff/.local/lib/python3.7/site-packages/flask_restful/init.py", line 269, in error_router return original_handler(e) File "/home/jeff/.local/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function return cors_after_request(app.make_response(f(*args, **kwargs))) File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 1866, in handle_exception reraise(exc_type, exc_value, tb) File "/home/jeff/.local/lib/python3.7/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app response = self.full_dispatch_request() File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/jeff/.local/lib/python3.7/site-packages/flask_restful/init.py", line 269, in error_router return original_handler(e) File "/home/jeff/.local/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function return cors_after_request(app.make_response(f(*args, **kwargs))) File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception reraise(exc_type, exc_value, tb) File "/home/jeff/.local/lib/python3.7/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request rv = self.dispatch_request() File "/home/jeff/.local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request return self.view_functionsrule.endpoint File "/home/jeff/.local/lib/python3.7/site-packages/flask_restful/init.py", line 458, in wrapper resp = resource(*args, **kwargs) File "/home/jeff/.local/lib/python3.7/site-packages/flask/views.py", line 89, in view return self.dispatch_request(*args, **kwargs) File "/home/jeff/.local/lib/python3.7/site-packages/flask_restful/init.py", line 573, in dispatch_request resp = meth(*args, **kwargs) File "/home/jeff/.local/lib/python3.7/site-packages/flask_jwt_extended/view_decorators.py", line 104, in wrapper return fn(*args, **kwargs) File "/home/jeff/.local/lib/python3.7/site-packages/flask_caching/init.py", line 414, in decorated_function rv = f(*args, **kwargs) File "/home/jeff/.local/lib/python3.7/site-packages/gramps_webapp/api.py", line 191, in get 'families': get_families(get_db()), File "/home/jeff/.local/lib/python3.7/site-packages/gramps_webapp/gramps.py", line 382, in get_families return {f.gramps_id: family_to_dict(db, f) for f in db.iter_families()} File "/home/jeff/.local/lib/python3.7/site-packages/gramps_webapp/gramps.py", line 382, in return {f.gramps_id: family_to_dict(db, f) for f in db.iter_families()} File "/home/jeff/.local/lib/python3.7/site-packages/gramps_webapp/gramps.py", line 247, in family_to_dict 'notes': get_note_ids(db, f), File "/home/jeff/.local/lib/python3.7/site-packages/gramps_webapp/gramps.py", line 230, in get_note_ids return [db.get_note_from_handle(h).gramps_id for h in x.get_note_list()] File "/home/jeff/.local/lib/python3.7/site-packages/gramps_webapp/gramps.py", line 230, in return [db.get_note_from_handle(h).gramps_id for h in x.get_note_list()] File "/usr/local/lib/python3.7/site-packages/gramps/gen/db/generic.py", line 1287, in get_note_from_handle return self._get_from_handle(NOTE_KEY, Note, handle) File "/usr/local/lib/python3.7/site-packages/gramps/gen/db/generic.py", line 1263, in _get_from_handle raise HandleError('Handle %s not found' % handle) gramps.gen.errors.HandleError: Handle e453189074973a4e2e58144f19c not found

[DavidMStraub]

Thanks for reporting! Apparently a family object has a note handle it reports via get_note_list but which does not actually exist. I didn't know that's possible. To better understand: do you set either GRAMPS_EXCLUDE_PRIVATE or GRAMPS_EXCLUDE_LIVING to True?

[ghost]

I do not set GRAMPS_EXCLUDE_PRIVATE or GRAMPS_EXCLUDE_LIVING to True. Although I'm not sure where I would set those variables. Are they environment variables? Is there any mention of these in the documentation?

[DavidMStraub]

Thanks, that eliminates two possible sources of error. (By the way, yes those are environment variables and they are mentioned in the Readme, which currently is the only documentation.)

Could you please run the "Check and Repair" tool in Gramps? (Tools > Database Repair > Check and Repair Database) and see if the problems persists after that?

I might have to add try/except clauses for each of the lookup functions otherwise.

[ghost]

How do I use the --force-unlock option?

I've tried several times, but can't get it to work.

ERROR: Use the --force-unlock option if you are sure that the database is not in use. ERROR: Use the --force-unlock option if you are sure that the database is not in use. ERROR: Use the --force-unlock option if you are sure that the database is not in use. ^Cjeff@jeff-OptiPlex-380 ~/projects/gramps-webapp $ gramps_webapp -O 'Family Tre2' run --force-unlock .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module named 'PyICU'. Localization will be impaired. Use your package manager to install PyICU Usage: gramps_webapp run [OPTIONS] Try "gramps_webapp run --help" for help.

Error: no such option: --force-unlock jeff@jeff-OptiPlex-380 ~/projects/gramps-webapp $

On 10/6/19 1:19 PM, David Straub wrote:

Thanks, that eliminates two possible sources of error. (By the way, yes those are environment variables and they are mentioned in the Readme, which currently is the only documentation.)

Could you please run the "Check and Repair" tool in Gramps? (Tools > Database Repair > Check and Repair Database) and see if the problems persists after that?

I might have to add try/except clauses for each of the lookup functions otherwise.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DavidMStraub/gramps-webapp/issues/17?email_source=notifications&email_token=ANM2UA7BI247DRG3MCZPOS3QNITZXA5CNFSM4I5ZSHYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOQOXA#issuecomment-538773340, or mute the thread https://github.com/notifications/unsubscribe-auth/ANM2UA3DNCSXCOHGDPI3UYTQNITZXANCNFSM4I5ZSHYA.

[ghost]

Ok. I ran the repair on the database, lots of issues. After that, I re-ran the gramps-webapp and gramps-webapp-frontend, and it appeared to work. I was able to get the app to run (first time after several weeks). Of course, this app probably needs to be more robust, but repair the database did work. thanks for your help.

[DavidMStraub]

OK good. Actually I'll keep the issue open until I've better understood where I need to be more failsafe w.r.t. broken links in the database (I wasn't really aware that this is possible).

Concerning force-unlock: this is an option of the Gramps CLI (which throws the error message) but isn't available for the web app. I could add it, but more likely I'll just change it such that it ignores the lock, since it's read-only anyway. In the meantime, you can use Gramps to break the lock, or simply delete the empty file lock in the database folder.