core icon indicating copy to clipboard operation
core copied to clipboard
verified publisher icon owncloud

[QA] occ file:scan silently fails when encountering a long filename

Open jnweiger opened this issue 2 years ago • 4 comments

Seen with core 10.13.0-beta.1 while trying to reproduce https://github.com/owncloud/core/pull/40726 (also reproducable with 10.12.2)

occ file:scan admin

 Scanning files for 1 users
 Starting scan for user 1 out of 1 (admin)

 +---------+-------+--------------+------------------+
 | Folders | Files | Elapsed time | Items per second |
 +---------+-------+--------------+------------------+
 | 8       | 11    | 00:00:00     | 22               |
 +---------+-------+--------------+------------------+

cd /var/www/owncloud/data/admin/files/Documents
cp ~/255_LoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIp.txt .
occ file:scan -vvv admin
 Scanning files for 1 users
 Starting scan for user 1 out of 1 (admin)
	Folder /admin/
	Folder /admin/cache
	Folder /admin/files
	Folder /admin/files/Photos
	File   /admin/files/Photos/Teotihuacan.jpg
	File   /admin/files/Photos/Portugal.jpg
	File   /admin/files/Photos/Lake-Constance.jpg
	Folder /admin/files/Documents
	File   /admin/files/Documents/Example.odt
	File   /admin/files/Documents/255_LoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIp.txt
	Folder /admin/files/Learn more about ownCloud
	File   /admin/files/Learn more about ownCloud/ownCloud Subscription Overview.pdf
	File   /admin/files/Learn more about ownCloud/Data Protection and Data Secrecy in ownCloud.pdf
	File   /admin/files/Learn more about ownCloud/Keep your files safe with ownCloud.pdf
	File   /admin/files/Learn more about ownCloud/Share files efficiently in your company.pdf
	File   /admin/files/Learn more about ownCloud/Safely leverage Microsoft Office.pdf
	File   /admin/files/Learn more about ownCloud/Introducing ownCloud.pdf

 +---------+-------+--------------+------------------+
 | Folders | Files | Elapsed time | Items per second |
 +---------+-------+--------------+------------------+
 | 8       | 12    | 00:00:00     | 25               |
 +---------+-------+--------------+------------------+

mysql owncloud -e "select path from oc_filecache where name like '%Lorem%';"
  • The new file has a filename with 255 characters. The filescan saw the file, and there was no error.
  • The total file count correctly increments by 1.
  • The file does not show up in oc_filecache. BAD.
cd ../Photos
mv Portugal.jpg PortuXXXXX.jpg
occ file:scan -vvv admin
 Scanning files for 1 users
 Starting scan for user 1 out of 1 (admin)
	Folder /admin/
	Folder /admin/cache
	Folder /admin/files
	Folder /admin/files/Photos
	File   /admin/files/Photos/Teotihuacan.jpg
	File   /admin/files/Photos/PortuXXXX.jpg
	File   /admin/files/Photos/Lake-Constance.jpg
	Folder /admin/files/Documents
	File   /admin/files/Documents/Example.odt
	File   /admin/files/Documents/255_LoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIp.txt
	Folder /admin/files/Learn more about ownCloud
	File   /admin/files/Learn more about ownCloud/ownCloud Subscription Overview.pdf
	File   /admin/files/Learn more about ownCloud/Data Protection and Data Secrecy in ownCloud.pdf
	File   /admin/files/Learn more about ownCloud/Keep your files safe with ownCloud.pdf
	File   /admin/files/Learn more about ownCloud/Share files efficiently in your company.pdf
	File   /admin/files/Learn more about ownCloud/Safely leverage Microsoft Office.pdf
	File   /admin/files/Learn more about ownCloud/Introducing ownCloud.pdf
	
 +---------+-------+--------------+------------------+
 | Folders | Files | Elapsed time | Items per second |
 +---------+-------+--------------+------------------+
 | 8       | 12    | 00:00:01     | 27               |
 +---------+-------+--------------+------------------+

mysql owncloud -e "select path from oc_filecache where name like 'Port%';"
 +---------------------------+
 | path                      |
 +---------------------------+
 | files/Photos/Portugal.jpg |
 +---------------------------+
  • Again, no error is shown. The rename never happened in the filecache. BAD
  • It is left to the poor admin to discern, why occ file:scan now fails, especially how he should locate the offending file in a real life scenario.

Expected behavior:

  • Print an error message, when a file cannot be proessed.
  • Continue processing all other files, instead of silently aborting.

jnweiger avatar Aug 16 '23 15:08 jnweiger

Workaround:

  • shorten the longest name to 250 chars. Then the file:scan works again.
mysql owncloud -e "select path from oc_filecache where name like '%Lorem%';"
 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 | path                                                                                                                                                                                                                                                                       |
 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 | files/Documents/250_IpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIpsumLoremIp.txt |
 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
mysql owncloud -e "select path from oc_filecache where name like 'Port%';"
 +----------------------------+
 | path                       |
 +----------------------------+
 | files/Photos/PortuXXXX.jpg |
 +----------------------------+

jnweiger avatar Aug 16 '23 15:08 jnweiger

Docker images owncloud/server:10.12.2 and owncloud/server:10.13.0-beta.1 are not affected. There a filescan happily brings a 255 char long name into the web UI

jnweiger avatar Aug 16 '23 16:08 jnweiger

No regression.

hodyroff avatar Aug 18 '23 09:08 hodyroff

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Feb 15 '24 01:02 github-actions[bot]

This issue has been automatically closed.

github-actions[bot] avatar Feb 25 '24 01:02 github-actions[bot]