sigma-file-manager icon indicating copy to clipboard operation
sigma-file-manager copied to clipboard

How long does global search take to search?

Open kleenkanteen opened this issue 4 years ago • 38 comments

Is there an existing issue for this?

  • [X] I have searched the opened issues

Problem description

I have only 500gb of files spread across 2 drives. It's been searching for 2 hours so far. On windows, cpu usage at constant 37% but disk usage is 0.1 MB/s. Is this normal? Now it just said error occured. I already reloaded it once then it searched for 2 hours. I will reload it again but not sure what is happening. Here is the error message, was the same as last time:

Error: UNKNOWN: unknown error, lstat 'C:\Users\Sabih\OneDrive\simple-react\node_modules\rollup\node_modules'

Steps to reproduce

Environment

System info:

  • App version: 1.4.0
  • App page: navigator
  • Operating System: x64, win32, 10.0.19043
  • Free memory: 6.3 GB

Problem:

Error:

Error: UNKNOWN: unknown error, lstat 'C:\Users\Sabih\OneDrive\simple-react\node_modules\rollup\node_modules'

App action history:

15:50:03:519 | App.vue::created()
15:50:03:670 | App.vue::mounted()
15:50:03:879 | store.js::UPDATE_STORAGE_FILE()
15:50:03:947 | App.vue::initAllStorageFiles()
15:50:04:198 | store.js::UPDATE_STORAGE_FILE()
15:53:25:466 | store.js::UPDATE_STORAGE_FILE()
16:13:47:881 | store.js::LOAD_DIR()
16:13:54:255 | store.js::UPDATE_STORAGE_FILE()

kleenkanteen avatar Sep 09 '21 20:09 kleenkanteen

Consider supporting the developer on Patreon to increase priority of all your issues, get access to private discord and other rewards.
Patreon page: https://www.patreon.com/sigma_file_manager

github-actions[bot] avatar Sep 09 '21 20:09 github-actions[bot]

Thanks for the report. That's definitely not normal. It usually takes around 1 minute to scan a 1 TB drive.

The problem is most likely caused by OneDrive. Other people reported problems with it as well. I will fix it for the next update.

Can you please download version 1.3, and see if the problem is still there?

https://github.com/aleksey-hoffman/sigma-file-manager/releases/tag/v1.3.0

aleksey-hoffman avatar Sep 09 '21 21:09 aleksey-hoffman

Same issue:

System info:

  • App version: 1.3.0
  • App page: settings
  • Operating System: x64, win32, 10.0.19043
  • Free memory: 7.3 GB

Problem:

Error:

Error: UNKNOWN: unknown error, lstat 'C:\Users\Sabih\OneDrive\easyreact\simple-react\node_modules\rollup\node_modules'

App action history:

17:30:55:672 | App.vue::created()
17:30:55:928 | App.vue::mounted()
17:30:56:033 | store.js::UPDATE_STORAGE_FILE()
17:30:56:069 | App.vue::initAllStorageFiles()
17:30:56:345 | store.js::UPDATE_STORAGE_FILE()
17:30:57:489 | store.js::UPDATE_STORAGE_FILE()
17:31:24:259 | store.js::UPDATE_STORAGE_FILE()
17:31:55:178 | store.js::UPDATE_STORAGE_FILE()

If I want to try fixing this what libraries will I need to get familiar wtih? Especially the ones that interact with the windows API?

kleenkanteen avatar Sep 09 '21 21:09 kleenkanteen

Hmm, I'm not sure but it seems like it didn't actually install v1.3 properly. Can you please click on the app icon in Tray and click "Close the app". And also please make sure there's no "Sigma File Manager" processes running in task manager

And then install the v1.3 again.

Do you see the same problem?

If I want to try fixing this what libraries will I need to get familiar wtih?

To fix such problems, you would probably need to have:

  • At least 6 months of development experience
  • Experience with all of these languages / frameworks: Electron.js, Node.js, Javascript, Vue.js

aleksey-hoffman avatar Sep 09 '21 21:09 aleksey-hoffman

Did that at the beginning btw, uninstalled the program even. Closed and uninstalled again, 1.3 version ofc, and as soon as I opened it it tried to auto update 1.4. As it was downloading the update I cancelled the download. This time I deleted the search data directory files. New error:

`## System info:

  • App version: 1.3.0
  • App page: settings
  • Operating System: x64, win32, 10.0.19043
  • Free memory: 7 GB

Problem:

Error:

Error: UNKNOWN: unknown error, lstat 'C:\Users\Sabih\OneDrive\easyreact\simple-react\node_modules\rollup\node_modules'

App action history:

18:12:05:959 | App.vue::created()
18:12:06:202 | App.vue::mounted()
18:12:06:373 | store.js::UPDATE_STORAGE_FILE()
18:12:06:406 | App.vue::initAllStorageFiles()
18:12:06:667 | store.js::UPDATE_STORAGE_FILE()
18:12:06:918 | store.js::UPDATE_STORAGE_FILE()
18:12:10:170 | store.js::UPDATE_STORAGE_FILE()
18:12:14:883 | store.js::UPDATE_STORAGE_FILE()
18:12:15:135 | store.js::UPDATE_STORAGE_FILE()
18:12:15:403 | store.js::UPDATE_STORAGE_FILE()
18:12:15:654 | store.js::UPDATE_STORAGE_FILE()
18:13:41:894 | store.js::LOAD_DIR()
18:13:47:084 | store.js::UPDATE_STORAGE_FILE()
18:13:47:970 | store.js::LOAD_DIR()
18:13:49:345 | store.js::LOAD_DIR()
18:13:50:430 | store.js::UPDATE_STORAGE_FILE()
18:13:50:468 | App.vue::initAllStorageFiles()
18:13:50:468 | store.js::LOAD_DIR()
18:13:50:719 | store.js::UPDATE_STORAGE_FILE()
18:13:52:372 | store.js::LOAD_DIR()
18:13:53:466 | store.js::UPDATE_STORAGE_FILE()
18:13:53:522 | store.js::LOAD_DIR()
18:13:53:593 | App.vue::initAllStorageFiles()
18:13:53:594 | store.js::LOAD_DIR()
18:13:53:846 | store.js::UPDATE_STORAGE_FILE()
18:13:54:615 | store.js::UPDATE_STORAGE_FILE()
18:13:54:667 | App.vue::initAllStorageFiles()
18:13:54:667 | store.js::LOAD_DIR()
18:13:54:913 | store.js::LOAD_DIR()
18:13:54:934 | store.js::UPDATE_STORAGE_FILE()
18:13:56:049 | store.js::UPDATE_STORAGE_FILE()
18:13:56:088 | App.vue::initAllStorageFiles()
18:13:56:339 | store.js::UPDATE_STORAGE_FILE()
18:14:00:414 | store.js::UPDATE_STORAGE_FILE()
18:14:18:576 | store.js::LOAD_DIR()
18:14:20:622 | store.js::LOAD_DIR()
18:14:25:631 | store.js::UPDATE_STORAGE_FILE()
18:14:26:185 | store.js::UPDATE_STORAGE_FILE()
18:14:26:206 | App.vue::initAllStorageFiles()
18:14:26:231 | App.vue::initAllStorageFiles()
18:14:26:481 | store.js::UPDATE_STORAGE_FILE()
18:14:39:712 | store.js::UPDATE_STORAGE_FILE()
18:14:39:743 | App.vue::initAllStorageFiles()
18:14:39:775 | App.vue::initAllStorageFiles()
18:14:40:408 | store.js::UPDATE_STORAGE_FILE()

Same issue whether **node_modules is in the ignored paths or not.

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Thanks, can you please open app's console with Ctrl + Shift + I and switch to the console tab. Then reload the app with Ctrl + Shift + R.

Do you see any errors there?

aleksey-hoffman avatar Sep 09 '21 22:09 aleksey-hoffman

Btw in the console as I am deleting search files, for every one I do it gives this error:

Uncaught (in promise) TypeError: Cannot read property 'name' of undefined at v.FETCH_INFO_PANEL_DATA (store.js:4088) at Array.<anonymous> (vuex.common.js:853) at v.dispatch (vuex.common.js:518) at Object.dispatch (vuex.common.js:408) at v.UPDATE_INFO_PANEL_DATA (store.js:4219) at Array.<anonymous> (vuex.common.js:853) at v.dispatch (vuex.common.js:518) at dispatch (vuex.common.js:408) at v.REPLACE_SELECTED_DIR_ITEMS (store.js:4052) at Array.<anonymous> (vuex.common.js:853)

So I reloaded the app again, and for global searching the error is now:

`### App action history:

18:29:24:649 | App.vue::created()
18:29:24:782 | App.vue::mounted()
18:29:25:053 | store.js::UPDATE_STORAGE_FILE()
18:29:25:149 | App.vue::initAllStorageFiles()
18:29:25:361 | store.js::UPDATE_STORAGE_FILE()
18:29:25:613 | store.js::UPDATE_STORAGE_FILE()
18:29:27:770 | store.js::UPDATE_STORAGE_FILE()

Error in console:

Uncaught Error: UNKNOWN: unknown error, lstat 'C:\Users\Sabih\OneDrive\simple-react\node_modules\rollup\node_modules'

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Thank you. I'm not sure what is causing the lstat error that you mentioned:

Uncaught Error: UNKNOWN: unknown error, lstat 'C:\Users\Sabih\OneDrive\simple-react\node_modules\rollup\node_modules'

Can you please reload the app again and when this error happens, expand the error message and then click on the top link that it points to.

And then show me what you see there. It should look something like this (the problematic line will be have red line under it)

image

image

aleksey-hoffman avatar Sep 09 '21 22:09 aleksey-hoffman

Btw, on Windows you can create screenshots with Win + Shift + S

aleksey-hoffman avatar Sep 09 '21 22:09 aleksey-hoffman

errorr

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Can you also make a screenshot of the console tab with that lstat error expanded?

aleksey-hoffman avatar Sep 09 '21 22:09 aleksey-hoffman

I tried expanding it but it doesn't expand, no option or arrow. Do you want to do a zoom call? I can do that in 20 minutes. 7:10 PM EST. I can send the link then.

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Oh actually I dont have onedrive open right now, let me start it and see.

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Same error after starting onedrive.

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Yeah, maybe we could connect via Zoom or Discord. Does either Zoom or Discord allow to give mouse / keyboard control to the other person? That would allow me to find the problem quickly

aleksey-hoffman avatar Sep 09 '21 22:09 aleksey-hoffman

Yup zoom allows remote access. See you in 19 minutes.

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

Actually I have something, can you do at 9 PM EST? If not choose a time after that till midnight.

kleenkanteen avatar Sep 09 '21 22:09 kleenkanteen

I'll be online for another 30 minutes at most. Maybe tomorrow then?

aleksey-hoffman avatar Sep 09 '21 22:09 aleksey-hoffman

Ok, will talk to you then.

kleenkanteen avatar Sep 09 '21 23:09 kleenkanteen

@kleenkanteen let me know when you can do this

aleksey-hoffman avatar Sep 11 '21 13:09 aleksey-hoffman

@aleksey-hoffman Sorry for the wait, will you be free in 10 minutes at 6:50? I can send the zoom link for it then.

kleenkanteen avatar Sep 12 '21 22:09 kleenkanteen

I'm on for a few more hours

kleenkanteen avatar Sep 12 '21 22:09 kleenkanteen

It's quite late for me right now. I'll be going to sleep soon. Maybe tomorrow? What's the earliest you can do?

aleksey-hoffman avatar Sep 12 '21 23:09 aleksey-hoffman

Probably 3pm est.

kleenkanteen avatar Sep 12 '21 23:09 kleenkanteen

I'm on for the next 10-12 hours. Let me know when you're up for it.

If I connect to your pc. I will probably need to launch the dev build of the app locally to find the problem, so if you could prep a few things that would great. so that I don't see any personal info you don't want me to see there accidentally while opening apps myself.

I will need:

  • Opened Github page of this app in the browser
  • File explorer window with some test folder where I can download the app and mess around with
  • A new code editor window, if you have any installed (VSCode preferably)

aleksey-hoffman avatar Sep 13 '21 10:09 aleksey-hoffman

Are you free at 12:50 EST? If so, I can drop the zoom link then.

kleenkanteen avatar Sep 13 '21 16:09 kleenkanteen

Let me know a good time for you

kleenkanteen avatar Sep 13 '21 16:09 kleenkanteen

I can get to it in 30 min - 1 hour from now. Is that good?

aleksey-hoffman avatar Sep 13 '21 16:09 aleksey-hoffman

What about now?

aleksey-hoffman avatar Sep 13 '21 17:09 aleksey-hoffman

Yup. You there?

kleenkanteen avatar Sep 13 '21 18:09 kleenkanteen

Yeah, send me the link in 5 minutes, and let's try to fix this problem

aleksey-hoffman avatar Sep 13 '21 18:09 aleksey-hoffman

So I was able fix the error. I got an error when right clicking on the faulty onedrive folder and selecting "Always keep on my drive/pc", for a folder inside it said "Data in the reparse buffer is invalid". This was the folder I think that sigma errored on. I googled that windows error and found this article https://social.technet.microsoft.com/Forums/en-US/11a0f585-b985-4a07-b6a0-f64dbce56536/onedrive-file-locked-with-quotdata-in-the-reparse-point-buffer-is-invalidquot-message?forum=win10itprogeneral. As admin in command prompt I wrote "chkdsk c: /F /R" and selected "run the next time my PC restarts". It ran for 10 minutes the next time I booted my pc and now sigma scanned my drives without issue.

I will try with 1.4 and see if it works there.

kleenkanteen avatar Sep 17 '21 07:09 kleenkanteen

@kleenkanteen good to know. I will have to fix the error on Sigma side for v1.5 update as well.

aleksey-hoffman avatar Sep 17 '21 11:09 aleksey-hoffman

I tried re scanning in 1.4, going on for 30 minutes, no error, but it is stuck on "scanning 2/5 drives". I will try again later this weekend.

kleenkanteen avatar Sep 17 '21 16:09 kleenkanteen

Yeah, that's not normal. It shouldn't take longer than 5 minutes in total for 5 drives. Is it showing any errors in the console during the scan?

aleksey-hoffman avatar Sep 17 '21 16:09 aleksey-hoffman

I might have to connect again to debug this, later

aleksey-hoffman avatar Sep 17 '21 16:09 aleksey-hoffman

Showing no errors in console. Are you on in about 17 hours, which is tomorrow at 12 pm EST? If so I will send you a link then for debugging.

kleenkanteen avatar Sep 17 '21 23:09 kleenkanteen

Showing no errors in console

Ok, that's a bit harder to debug, but it's fine, we'll figure it out. I'm not sure about the time, let's decide tomorrow

aleksey-hoffman avatar Sep 17 '21 23:09 aleksey-hoffman