OmniParser icon indicating copy to clipboard operation
OmniParser copied to clipboard

FIx this damn thing

Open paciox opened this issue 6 months ago • 10 comments

It's getting ridiculous fix this broken toy.

  1. It's time, you have to update the guide to run local models and also support more of them, better if with Ollama
  2. It's complicated to install. You require a powerful machine and a ton of passages with 3 separated components. Do a goddamn step by step full guide for win and linux.
  3. It breaks. World goes on and people at QEMU decided to take it down from docker. Now installation breaks and some chinese people suggest obscures docker repos, also in the Dockerfile it needs wsdd2 and not "wsdd" that now breaks.
  4. It needs goddamn indicators if it's running. when running the script to create the windows vm it says continuously "Waiting for a response from the computer control server. When first building the VM storage folder this can take a while..." and we don't know if it's broken/crashed or it's ok. 5.It throws random errors. i.E [+] Running 1/1 ✘ windows Error pull access denied for windows-local, repository does not exist or may require ...

Then you restart and it goes and so on.

6.Even after fixing all this mess, it still stays in the console outputting "Waiting for a response from the computer control server. When first building the VM storage folder this can take a while... Waiting for a response from the computer control server. When first building the VM storage folder this can take a while... Waiting for a response from the computer control server. When first building the VM storage folder this can take a while... Waiting for a response from the computer control server. When first building the VM storage folder this can take a while... Waiting for a response from the computer control server. When first building the VM storage folder this can take a while..."

It's been there for the last good 4 hours so I don't think it's a problem of waiting anymore. This log message doesn't have a progress bar, an error log or anything to understand what is happeing.... come on.....

FIXXXXXXXX

paciox avatar Jul 07 '25 00:07 paciox

Managed to solve previous problems, now I am facing the infamous: 2025-07-07 16:50:12 omni-windows | ❯ Warning: the filesystem of /storage is FUSE, this extra layer will negatively affect performance! 2025-07-07 16:50:12 omni-windows | /usr/bin/env: 'python3': No such file or directory 2025-07-07 16:50:12 omni-windows | ❯ ERROR: KVM acceleration not available (device file missing), this will cause a major loss of performance. 2025-07-07 16:50:12 omni-windows | ❯ ERROR: See the FAQ on how to diagnose the cause, or continue without KVM by setting KVM=N (not recommended).

Windows doesn't see kvm and I need to find a solution, as, from what I read in the message but also in the issues, KVM=N will brutally slow down the windows machine and so it's not really a solution. Moreover also python is missing.

paciox avatar Jul 07 '25 16:07 paciox

I tried everything to set KVM up in windows also asking AI and FAILED. I activated every windows feature about virtualuzation (Hyper-V, Virtual Machine Platform, WSL2) as indicated by AI, I activated it in BIOS and everything else. Nothing worked. Sadly I resorted to the infamous KVM="N" and now this happens #295

This stuff is broken to the bone

paciox avatar Jul 09 '25 07:07 paciox

@paciox buddy have you found a solution? I m facing the same issue

end-9214 avatar Aug 09 '25 09:08 end-9214

@end-9214 I've already made it work. I installed a virtual machine from Debian 12 with Hyper-V on Windows, then installed Win11 with Docker in Debian 12.

guangxuanliu avatar Aug 15 '25 02:08 guangxuanliu

@end-9214 I've already made it work. I installed a virtual machine from Debian 12 with Hyper-V on Windows, then installed Win11 with Docker in Debian 12.

Ooh cool; how much time did it took to install all the apps and setup?

end-9214 avatar Aug 15 '25 03:08 end-9214

@end-9214 I've already made it work. I installed a virtual machine from Debian 12 with Hyper-V on Windows, then installed Win11 with Docker in Debian 12.

Ooh cool; how much time did it took to install all the apps and setup?

There are many issues inside, and it might take a few days.

guangxuanliu avatar Aug 15 '25 03:08 guangxuanliu

@end-9214 I've already made it work. I installed a virtual machine from Debian 12 with Hyper-V on Windows, then installed Win11 with Docker in Debian 12.

Ooh cool; how much time did it took to install all the apps and setup?

There are many issues inside, and it might take a few days.

After you fixed the VM it took few days to install ?? Damn

end-9214 avatar Aug 15 '25 03:08 end-9214

If Docker is available, you can use https://github.com/microsoft/OmniParser/pull/52/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557

abrichr avatar Aug 18 '25 20:08 abrichr

If Docker is available, you can use https://github.com/microsoft/OmniParser/pull/52/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557

is it just for omni parser or also for omnitool? because omniparser is working fine ; i m facing problems with omnitool

end-9214 avatar Aug 20 '25 08:08 end-9214

@paciox buddy have you found a solution? I m facing the same issue

Unfortunately no. The only viable way at the moment seems to give it a shot on linux, but I will face even more troubles because I don't want to use an online paid model, rather a local setup. Once I find enough courage, I will try a linux setup. The guy below me seems to have made it, even though I'm really curious and skeptical of what are the performance of a Windows VM nested inside a Debian 12 VM on a host machine, since this also requires you to have a powerful GPU correctly channeled through multiple layers of virtualization. unless he installs the 2 server models using them via network, since I saw that the python launcher given with the repo has arguments and can be launched as follows:

python app.py –windows_host_url localhost:8006 –omniparser_server_url localhost:8000

Which means I would be fine by having the windows nested machine on my local pc (No good gpu) and the AI services on a rented GPU service reachable via network.

So this looks like it works?

@end-9214 I've already made it work. I installed a virtual machine from Debian 12 with Hyper-V on Windows, then installed Win11 with Docker in Debian 12.

But it still requires quite some setup and further fiddling with code/setup/networking/vm configuration to say I can really do something.

And this whole thing they released is the definition of walking on eggshells. It literally can break at any step and leave you in despair.

The real solution would seem like getting expertise in the setup they made with BSD.exe and all the python scripting, and totally readapting it with the new libraries and correct configuration to make a whole new repo.

If only the guy who did this (which is some asian researcher) didn't threw it here like an experimental graduation project and abandoned it completely....

That said:

@end-9214 I've already made it work. I installed a virtual machine from Debian 12 with Hyper-V on Windows, then installed Win11 with Docker in Debian 12.

Ooh cool; how much time did it took to install all the apps and setup?

There are many issues inside, and it might take a few days.

Why you don't describe them to use and write also the solution? So we don't waste time.

Also, are you running Omniparser or Omniparser+Omnitool so you can control windows? (I guess the latter since you installed a windows vm)

Can we see it in action? Can you do a snapshot of the vm in clean state with no personal information and give us the docker/compose/yaml file and the whole setup? What is your hardware?

If Docker is available, you can use https://github.com/microsoft/OmniParser/pull/52/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557

is it just for omni parser or also for omnitool? because omniparser is working fine ; i m facing problems with omnitool

From a quick look seems just the omniparser thing. I was already able to run it on a local pc and a google colab on free tier.

Omnitool is a different beast.

paciox avatar Aug 20 '25 09:08 paciox