crash when using data file from 0.3.1

[paulgear]

I recently upgraded from Ubuntu 20.04 to 22.04, and gtg 0.3.1 seems to have been removed from their repos. I cloned from git, added the dependencies, and I can run the default data set no problem. However, when I run ./scripts/import_tasks_from_local.sh, it doesn't work:

1. If I just leave the data files where they are, the uuid-named XML file gets copied into tmp/default/xdg/data/gtg/, but none of my previous tasks show up in the list
2. If I copy the uuid-named XML file over the top of gtg_data.xml, gtg crashes with the following traceback on startup:

Context: Startup

Traceback (most recent call last):
  File "/home/paulgear/src/getting-things-gnome/gtg/.local_build/install/lib/python3/dist-packages/GTG/gtk/application.py", line 111, in do_startup
    datastore.register_backend(backend_dic)
  File "/home/paulgear/src/getting-things-gnome/gtg/.local_build/install/lib/python3/dist-packages/GTG/core/datastore.py", line 459, in register_backend
    source.initialize(connect_signals=False)
  File "/home/paulgear/src/getting-things-gnome/gtg/.local_build/install/lib/python3/dist-packages/GTG/core/datastore.py", line 816, in initialize
    self.backend.initialize()
  File "/home/paulgear/src/getting-things-gnome/gtg/.local_build/install/lib/python3/dist-packages/GTG/backends/backend_localfile.py", line 136, in initialize
    self.datastore.load_tag_tree(self.tag_tree)
  File "/home/paulgear/src/getting-things-gnome/gtg/.local_build/install/lib/python3/dist-packages/GTG/core/datastore.py", line 232, in load_tag_tree
    for element in tag_tree.iter('tag'):
AttributeError: 'NoneType' object has no attribute 'iter'

Software versions:

• GTG (debug version) v0.6
• CPython 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
• GTK 3.24.33, GLib 2.71.3
• PyGLib 3.42.1, PyGObject 3.42.1
• Linux-5.15.0-43-generic-x86_64-with-glibc2.35

[diegogangl]

Did you try renaming the uuid file to gtg_tasks.xml? Also don't forget to copy tags.xml and project.xml (if it exists) If it's called gtg_data gtg won't run the 0.3 versioning

[paulgear]

Did you try renaming the uuid file to gtg_tasks.xml? Also don't forget to copy tags.xml and project.xml (if it exists) If it's called gtg_data gtg won't run the 0.3 versioning

Yes, that's how the above crash happens. If I don't move or copy the uuid file to gtg_tasks.xml, gtg proceeds with the default dataset as if I had never run scripts/import_data_from_local.sh.

[diegogangl]

But did you also copy tags.xml? (sorry for the delays -_-)

[paulgear]

But did you also copy tags.xml? (sorry for the delays -_-)

Yep, that's handled automatically by scripts/import_data_from_local.sh, and I confirmed that it's there and looks correct. If I don't copy/rename the uuid file over gtg_tasks.xml, it seems like it never realises a conversion is needed. Here are the last lines on standard output before the GUI shows:

Installing /home/paulgear/src/getting-things-gnome/gtg/docs/user_manual/C/figures/workview.png to /home/paulgear/src/getting-things-gnome/gtg/.local_build/install/share/help/C/gtg/figures
2022-08-18 07:04:57,582 - ERROR - datastore2:save:152 - Error renaming temp file '/home/paulgear/src/getting-things-gnome/gtg/tmp/default/xdg/data/gtg/gtg_data.xml__'
2022-08-18 07:05:00,656 - WARNING - adaptive_button:do_forall:279 - Got error in for but it should've stay valid: AttributeError("'AdaptiveFittingWidget' object has no attribute '_children'")
2022-08-18 07:05:00,667 - ERROR - datastore2:save:152 - Error renaming temp file '/home/paulgear/src/getting-things-gnome/gtg/tmp/default/xdg/data/gtg/gtg_data2.xml__'

[diegogangl]

Could you try deleting tmp/default/xdg/config?

[paulgear]

Could you try deleting tmp/default/xdg/config?

No difference, sorry. :cry:

[diegogangl]

Could you send me the tags and projects files to [email protected]? You can change the names of the tags if you want to anonymize it a bit (unfortunately our anonymize script only works with the new xml files).

[paulgear]

It's got pretty much my whole life in it, including data about work clients, so that's not really an option. I'll take some time trying to get up to speed with the codebase and see what I can work out.

[diegogangl]

I meant only the tags.xml and projects.xml files (not the tasks). No problem though, let me know if you have any questions o/

[paulgear]

Ah, sorry - I misunderstood. Here are the files, with tags anonymised. gtg-files.zip

[paulgear]

OK, so I got my development environment sorted out enough (thanks to dgangl & Neui on IRC/Matrix) to launch it in the VS Code debugger and dug into the code. Here's what I worked out:

1. When versioning.py:is_required() is called, it hard-codes the old filename as gtg_tasks.xml, and the new file is passed in as gtg_data.xml (both with a DATA_DIR prefix). This doesn't work because:

   • the old files in my case (copied into place by scripts/import_tasks_from_local.sh) are projects.xml, tags.xml, and UUID.xml
   • new file is already present because the backend startup saves it (see datastore2.py:first_run())

2. versioning.py:is_required() and backend_localfile.py:this_is_the_first_run() independently hard-code the old file as gtg_tasks.xml. After reviewing the other code in versioning.py, I came to the conclusion that it really is about XML conversion (and probably should be renamed to that effect) rather than the decision to run the conversion or not.

3. backend_localfile.py has two different blocks of near-identical code, in this_is_the_first_run() and initialize(), and the latter runs after the former, so there really isn't any need for the second copy of it.

4. Even if we start with the correct local file, there seems to be no logic that opens up projects.xml to find the name of the UUID file where the tasks are stored.

So I've created a branch which fixes these issues and has allowed me to successfully convert my data file (which was created by version 0.3.1-4 on Ubuntu 20.04). I'll submit a PR to link it with this issue shortly.