files_scripts Rescan files with occ after a file operation is done

Hi! I use now nc v.28. I've created an app to unzip an archive file.

A piece of script:

local input_file = get_input_files()[1]
local selected_files = #get_input_files()
local dir = get_input("output_location")
local supported_formats = { 'zip' }
local input_file_extension = input_file.name:match("[^.]+$")
...
local input_path = meta_data(input_file).local_path
local output_path = meta_data(dir).local_path
...
command_tpl = 'unzip "%s" -d "%s"'
command = command_tpl:format(input_path, output_path)
....

So, the script unzips the file: I see it created in the server files structure, but the file doesn't appear in the webUI. Am I forgetting to add some function to rescan files? Please, advice.

May 21 '24 22:05 oleua

@Raudius UPD: I found a workaround:

-- rescan folder after a file operation is done
local output_storage_path = meta_data(dir).storage_path
command_tpl = 'php <path_to_occ>/occ files:scan --path="%s"'
command = command_tpl:format(output_storage_path)
result = shell_command(command)

if result.exit_code ~= 0 then 
  add_message("Error: " .. result.errors,"error")
end

This will let show files in webUI, and their actual size after some file operations are done.

Perhaps some better way may be used? What do you think?

May 26 '24 06:05 oleua

Great to hear you have found a workaround for now

I will look but I dont think there is a public API from nextcloud to rescan files

May 31 '24 10:05 Raudius

Great to hear you have found a workaround for now

I will look but I dont think there is a public API from nextcloud to rescan files

Rauidius, there should be 2 operations:

rescan files/folders in the directory where changes happened
refresh the webpage with webUI

It is interesting, that in some scripts files appear in webUI, while in others, they do not. The same is true for the size of the files.

Would you mind to create a feature request if the mentioned API is not present yet.

Jun 01 '24 14:06 oleua

For refreshing a page, can I use this snippet?

page = [[ 
<head><meta http-equiv="refresh" content="0"></head> 
]]
ngx.say(page)

Jun 03 '24 09:06 oleua

I've tried with

local refresh_page = 'header(\"Refresh:0\");'
command_tpl = "php -r '%s'"
command = command_tpl:format(refresh_page)
result= shell_command(command)

But that has resulted the error without any description. Perhaps, you may suggest, what to do to refresh the current webpage after executing all previous commands.

Jun 10 '24 10:06 oleua

The file list is always refreshed after completing a file action. I suspect the problem is that the occ command takes some time for the scan to happen, you can try experimenting with wait after the rescan

Something like:

-- rescan folder after a file operation is done
local output_storage_path = meta_data(dir).storage_path
command_tpl = 'php <path_to_occ>/occ files:scan --path="%s"'
command = command_tpl:format(output_storage_path)
result = shell_command(command)

if result.exit_code ~= 0 then 
  add_message("Error: " .. result.errors,"error")
end

wait(5)

Jun 10 '24 11:06 Raudius

wait(5)

Unfortunately the file list is not refreshed automatically after the operation is done.

Here is the full list of the script:

-- Archive files with the help of 7z and rar
local input_file = get_input_files()[1]
local selected_files = #get_input_files()
local archiver = get_input("selected_archiver")
local password = get_input("archive_password")
local dir = get_input("output_location")
local supported_formats = { 'zip', '7z', 'rar', 'gzip', 'bzip', 'bzip2', 'tar' }
local input_file_extension = input_file.name:match("[^.]+$")

-- Function to check file presence of system apps
local function file_exists(name)
   local f=io.open(name,"r")
   if f~=nil then io.close(f) return true else return false end
end

-- Function to check whether a table contains some element
local function table_contains(table, element)
    for _, value in pairs(table) do
        -- We grab the first index of our sub-table instead
        if value == element:lower() then
            return true
        end
    end
    return false
end

local function table_not_contains(table, element)
    for _, value in pairs(table) do
        -- We grab the first index of our sub-table instead
        if value == element:lower() then
            return false
        end
    end
    return true
end

-- Function to search substring

local function get_lines_with_text(inputstr, search)
    local t={}
    for str in string.gmatch(inputstr, "[^\n]*"..search.."[^\n]*") do
        table.insert(t, str)
    end
    return t
end

-- Checking presence of system apps to convert files

if ((not file_exists('/usr/bin/7za')) and (not file_exists('/usr/local/bin/7za'))) then 
  	abort("Потрібно встановити програму p7zip.")
end

if ((not file_exists('/usr/bin/rar')) and (not file_exists('/usr/local/bin/rar'))) then 
  	abort("Потрібно встановити програму rar.")
end

-- Check if not selected more than 1 file
if (selected_files > 1) then abort("Вибрати для архівування поки що можна тільки один файл або каталог.") end

-- Defining default output format ZIP
if (archiver == nil or archiver == "") then
	archiver = "zip"
end

-- Creating current date stamp using current timezone of the server
now = create_date_time()
result = shell_command('date +%z')
timezone = "GMT" .. result.output
format = "ddMMyyyy_HHmm"
current_date_stamp = format_date_time(now, nil, timezone, format)

-- Checking that the output location is defined
if (dir == nil or dir == "") then
   dir={}
   dir.path = input_file.path
   dir.name = ""
end

-- Check if the selected item is a folder or it is a file
if (is_folder(input_file) == true) then

-- Defining that the output_name is the same as the input name but with archiver extension 
  output_name = input_file.name .. "." .. archiver

-- Defining output_name with current date stamp
  output_name_stamp = input_file.name .. "_" .. current_date_stamp .. "." .. archiver
end

if (is_folder(input_file) == false) then
-- Defining that the output_name is the same as the input name but with archiver extension
  output_name = input_file.name:match("(.+)%..+") .. "." .. archiver

-- Defining output_name with current date stamp
  output_name_stamp = input_file.name:match("(.+)%..+") .. "_" .. current_date_stamp .. "." .. archiver

-- Checking that the destination file exists with the same extension, if yes, add date fingerprint
end

-- Add timestamp if the current file exists
if (exists(dir,output_name)) then 
   	output_name = output_name_stamp
end

-- Defining output_file
-- local output_file = new_file(dir, output_name)

-- Adding full paths
local input_path = meta_data(input_file).local_path
local output_path = meta_data(dir).local_path .. '/' .. output_name

-- Processing files
if archiver ~= 'rar' then
  if (password == nil or password == "") then
	command_tpl = '7za -t%s a "%s" "%s"'
    command = command_tpl:format(archiver, output_path, input_path)
  else
  	command_tpl = '7za -p%s -t%s a "%s" "%s"'
    command = command_tpl:format(password, archiver, output_path, input_path)
  end
else
  if (password == nil or password == "") then
	command_tpl = 'rar -ep a "%s" "%s"'
    command = command_tpl:format(output_path, input_path)
  else
    command_tpl = 'rar -p%s -ep a "%s" "%s"'
    command = command_tpl:format(password, output_path, input_path)
  end
end

result = shell_command(command)

-- Handle error
if result.exit_code == 0 then
  add_message("Успішно створено архів. Оновіть сторінку!", "info")
else 
  abort("Помилка: " .. result.errors)
end

-- Postprocess to rescan folder with created file
local output_storage_path = meta_data(dir).storage_path
command2_tpl = 'php /usr/local/bin/occ files:scan --path="%s"'
command2 = command2_tpl:format(output_storage_path)
result = shell_command(command2)

-- Handle result
if result.exit_code ~= 0 then 
  add_message("Помилка: " .. result.errors,"error")
end

-- Wait while scanned and the page refreshed
wait(5)

Jun 10 '24 19:06 oleua

I just noticed the wait function is broken (it only halts for 1 second), can you experiment with multiple waits?

Something like:

-- Wait while scanned and the page refreshed
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)

Jun 10 '24 19:06 Raudius

wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)
wait(1)

Nope, that didn't help. The file list wasn't refreshed.

Jun 10 '24 19:06 oleua

Does manually reloading the page after the script execution actually show the files? Because the file list is always updated after the script execution:

https://github.com/Raudius/files_scripts/blob/master/src/views/ScriptSelect.vue#L146

The occ scan just takes some time asynchronously to finish the scanning. Maybe try adding more wait(1) at the end of the script and see how many until you can see the files.

Jun 10 '24 19:06 Raudius

Yes, when refreshed manually the file appears. Please, see this video. At 00:53 I just click Ctrl-R in my browser and the archived file appears.

https://github.com/Raudius/files_scripts/assets/5199030/c4ddad9f-ad1f-418b-b6f2-99c4bf3a3e40

I use NC 28.0.6 Apache, PHP 8.2

Jun 10 '24 19:06 oleua

Maybe the issue is with the script itself? Eg, I am not opening new file. I've also experimented with the key 'filesystem_check_changes' => 1|0 in config.php, however without any results.

Jun 10 '24 20:06 oleua

You can check if the filesystem reload is happening by looking at the network log (F12 > Network), there should be a PROPFIND call to remote.php after the script execution.

Also, what Nextcloud version are you on?

Jun 10 '24 20:06 Raudius

You can check if the filesystem reload is happening by looking at the network log (F12 > Network), there should be a PROPFIND call to remote.php after the script execution.

Also, what Nextcloud version are you on?

NC 28.0.6

When starting the script and before clicking the button to initiate some operation, the PROPFIND appears. However, after the script execution nothing new appears in the logs till the web page is manually refreshed.

Jun 10 '24 20:06 oleua

@Raudius Hi! Are any ideas, how to fix that?

Jun 11 '24 16:06 oleua

@Raudius Sorry to bother you. Are any chances to fix this issue?

Jul 05 '24 15:07 oleua