Show extract status more prominently.

[koutheir]

Description

Clicking on the Extract... button does not behave as illustrated in the documentation.

Steps to reproduce:

• Start vorta in the terminal.
• Open the varta window from the notification icon.
• Choose the last archive backed up.
• Click the Extract... button.
• Wait until getting archive content... is no more displayed in the status line.
• Observe Done getting archive content in the status line.
• Nothing else happens. No windows are displayed. No errors are displayed. It's as if the operation succeeded somehow without any user input.
• Quit vorta (from the notification icon menu).
• Ctrl+C in the terminal, as vorta takes too long to close.

Reproduction

• [X] I tried to reproduce the issue.
• [X] I was able to reproduce the issue.

OS

Debian Linux Bookworm, KDE Plasma 5.26.5

Version of Vorta

0.8.10

What did you install Vorta with?

Pip

Version of Borg

1.2.3

Logs

...
2023-01-29 18:16:04,700 - vorta.views.source_tab - DEBUG - Added item number 0 from 1
2023-01-29 18:16:04,700 - vorta.views.source_tab - DEBUG - Added item number 1 from 2
2023-01-29 18:16:04,701 - vorta.views.source_tab - DEBUG - Added item number 2 from 3
2023-01-29 18:16:04,785 - root - INFO - Using NetworkManagerMonitor NetworkStatusMonitor implementation.
2023-01-29 18:16:04,819 - vorta.borg.jobs_manager - DEBUG - Add job for site default
2023-01-29 18:16:04,820 - vorta.borg.jobs_manager - DEBUG - Start job on site: default
2023-01-29 18:16:04,824 - vorta.borg.borg_job - INFO - Running command /usr/bin/borg --version
2023-01-29 18:16:05,127 - vorta.borg.jobs_manager - DEBUG - Finish job for site: default
2023-01-29 18:16:05,127 - vorta.borg.jobs_manager - DEBUG - No more jobs for site: default
2023-01-29 18:16:05,127 - vorta.scheduler - DEBUG - Refreshing all scheduler timers
2023-01-29 18:16:05,128 - vorta.scheduler - DEBUG - Scheduler for profile 1 is disabled.
2023-01-29 18:16:15,001 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-01-29 18:16:15,022 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-01-29 18:16:15,023 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2023-01-29 18:16:15,023 - vorta.borg.borg_job - DEBUG - Password not found in primary keyring. Falling back to VortaDBKeyring.
2023-01-29 18:16:15,028 - vorta.borg.jobs_manager - DEBUG - Add job for site 1
2023-01-29 18:16:15,028 - vorta.borg.jobs_manager - DEBUG - Start job on site: 1
2023-01-29 18:16:15,043 - vorta.borg.borg_job - INFO - Running command /usr/bin/borg list --info --log-json --json-lines --format {mode}{user}{group}{size}{isomtime}{path}{source}{health}{NL} <...>/laptop-data::hw2494-2023-01-24-182200
2023-01-29 18:19:57,306 - vorta.borg.jobs_manager - DEBUG - Finish job for site: 1
2023-01-29 18:19:57,307 - vorta.borg.jobs_manager - DEBUG - No more jobs for site: 1
2023-01-29 18:19:57,307 - vorta.scheduler - DEBUG - Scheduler for profile 1 is disabled.
2023-01-29 18:21:46,832 - vorta.borg.borg_job - DEBUG - Cancel job on site default
2023-01-29 18:21:46,832 - vorta.borg.borg_job - DEBUG - Cancel job on site 1
2023-01-29 18:21:46,832 - vorta.borg.jobs_manager - INFO - Finished cancelling all jobs
2023-01-29 18:21:46,905 - root - CRITICAL - Uncaught exception, file a report at https://github.com/borgbase/vorta/issues/new/choose
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/vorta/views/extract_dialog.py", line 39, in run
    parse_json_lines(lines, self.model)
  File "/usr/local/lib/python3.11/dist-packages/vorta/views/extract_dialog.py", line 215, in parse_json_lines
    model.addItem(
  File "/usr/local/lib/python3.11/dist-packages/vorta/views/partials/treemodel.py", line 370, in addItem
    self.beginResetModel()
RuntimeError: wrapped C/C++ object of type ExtractTree has been deleted
^C
fish: Job 1, 'vorta' terminated by signal SIGSEGV (Address boundary error)

[real-yfprojects]

In the archive tab there should be a text saying Processing archive contents. Do you see that?

[koutheir]

Do you mean somewhere in the status bar?

[koutheir]

Yes, I do see Processing archive contents right below the refresh button. htop shows the vorta process running 100%. But the window itself doesn't show any signs of being busy doing something, and the Extract... button is also available.

[koutheir]

The vorta process is still running at 100% and consuming 23 GiB of RSS memory. I see no changes in the vorta window.

[koutheir]

After 4 hours, I got a window asking to Choose files to extract. It looks like this feature works, but nothing was asking me to wait or informing me that this operation may take a long time to finish.

[real-yfprojects]

After 4 hours, I got a window asking to Choose files to extract. It looks like this feature works, but nothing was asking me to wait or informing me that this operation may take a long time to finish.

Still this is very bad performance for showing a list of files. Do you have an estimate of the file count? It would be nice if you could compare the performance to an earlier vorta version (<0.8.8).

[koutheir]

Do you have an estimate of the file count?

Which count? total backed up? Or files changed in that particular archive?

[real-yfprojects]

Which count? total backed up? Or files changed in that particular archive?

Total number of files in the archive

[koutheir]

I think it is around 7 million files total backed up.

[m3nu]

The Extract window isn't made for this number of files, since they need to be kept in memory. You'll want to use the Borg CLI to extract this.

[real-yfprojects]

I was thinking maybe there is a way to load directories from an archive on demand. Borg list supports a PATH argument and pattern exclusion.

@m3nu This is still an enhancement, see title.

[ganeshrevadi]

I can work on this issue , Can someone clarify what has to be done and the name of the files in which the changes has to be made!

[real-yfprojects]

Processing archive contents isn't displayed in the statusbar, where other progress messages are shown but at a different in the archive tab. That has to be changed.

[diivi]

Should this be done with all _set_status calls (with all actions including processing archive contents like No archive selected etc.?

[real-yfprojects]

Should this be done with all _set_status calls (with all actions including processing archive contents like No archive selected etc.?

I am not entirely sure, but most of them would benefit from such a change.

[m3nu]

Yes, this secondary status box isn't great. I wanted to get rid of it for a while. It's only used for mounting and a few other actions.

[keysmashes]

The Extract window isn't made for this number of files, since they need to be kept in memory.

A warning or error if the file count exceeds some limit (1 million files? 100,000?) would be perhaps a good idea. (I have probably around 12 million files backed up and was wondering how long I would need to wait...)