Support Controlling PTZ Cameras Via WebUI

[NickM-27]

This is a WIP for a future version

Open Questions:

• Do we want to force detection disabled while camera is moving?

To-Do:

• [x] Handle basic ptz commands
• [x] HTTP endpoint with current ptz status
• [x] Frontend to move camera
• [x] Frontend to zoom camera
• [x] Frontend to move to camera presets
• [x] Handle auth errors
• [x] Don't connect to onvif when camera is disabled
• [x] Keyboard shortcuts
• [x] fix mobile long press
• [x] Add all ptz cameras to one control panel in birdseye view
• [x] Update docs

[netlify[bot]]

✅ Deploy Preview for frigate-docs ready!

Name	Link
🔨 Latest commit	997d4c0a4d0d964e2a15210a0e9ce834d0d4973d
🔍 Latest deploy log	https://app.netlify.com/sites/frigate-docs/deploys/64467763ed625800085828a2
😎 Deploy Preview	https://deploy-preview-4715--frigate-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[felipecrs]

Wow, really cool. This reminds me of https://github.com/blakeblackshear/frigate/pull/2333.

I like that this is introducing ONVIF to Frigate. This certainly will make possible many features for the feature, like discovering cameras and camera streams perhaps.

[NickM-27]

Fun demo of the web UI from some work I did this morning. Keep in mind that the stream is MSE so webRTC would be even more responsive

https://user-images.githubusercontent.com/14866235/207948329-253fc763-dde1-45f9-ba66-f566598b3bb4.mov

[banthungprong]

Until now I have to control my PTZ camera via a script (based on this https://github.com/agsh/onvif) or using the Web Ui of the camera (start UI in Browser, Login, expand PTZ Control, use it). This will be a great extension for frigate which is running permanently on my PC. Question: Will keyboard shortcuts work?

[NickM-27]

Question: Will keyboard shortcuts work?

No reason they couldn't

[cyberstrawberry101]

Certainly looking forward to PTZ support of any kind so I can more readily get away from Blue Iris. I will be keeping an eye on this PR.

[felipecrs]

@NickM-27 quick question, is your goal to have this PR included in 0.12 still?

[NickM-27]

@NickM-27 quick question, is your goal to have this PR included in 0.12 still?

I never intended for this to be included in 0.12

[felipecrs]

Got it. Thanks.

Just thinking aloud: perhaps you can consider including this feature behind the use_experimental flag? No need to answer. :)

[cyberstrawberry101]

This is moving along quickly. Even rudimentary PTZ under some kind of environment flag would be huge for the early-adopters who want to use it.

I deploy Frigate as a docker container, so something like an environment variable "EXPERIMENTAL_BUILD=true" would be more than okay on my side of the fence, and just hide the WebUI elements for folks that do not have that variable enabled.

[felipecrs]

Hey @NickM-27, I did not check the PR in details, but I'd like to give you an idea (in case you didn't have it already).

A way to specify for the camera which capabilities it has about PTZ (so that, for example, the buttons for the unsupported capabilities aren't shown). I have a PTZ camera, but it doesn't support zoom, neither presets. It only supports pan/tilt, in Continuous Move mode.

https://user-images.githubusercontent.com/29582865/214072017-1ca599ff-1afa-41cf-9c65-bb8af700a3f5.mp4

[NickM-27]

Hey @NickM-27, I did not check the PR in details, but I'd like to give you an idea (in case you didn't have it already).

A way to specify for the camera which capabilities it has about PTZ (so that, for example, the buttons for the unsupported capabilities aren't shown). I have a PTZ camera, but it doesn't support zoom, neither presets. It only supports pan/tilt, in Continuous Move mode.

https://user-images.githubusercontent.com/29582865/214072017-1ca599ff-1afa-41cf-9c65-bb8af700a3f5.mp4

Yes, it already does this

[NickM-27]

Example of birdseye view:

[kirsch33]

@NickM-27 just curious when this would be merged into a release? from the screenshots youve posted it looks really good. would love to have this feature for some Wyze cams that I plan to flash the rtsp firmware to.

[NickM-27]

@NickM-27 just curious when this would be merged into a release? from the screenshots youve posted it looks really good. would love to have this feature for some Wyze cams that I plan to flash the rtsp firmware to.

Like I said above, this won't be making it into 0.12

[ehn]

@NickM-27 Have you given any thought to how to handle zones for PTZ cameras? Panning, tilting and zooming would, of course, make any hard-coded zone definition useless.

It would be awesome if Frigate could automatically recompute the zone definition, but that's a non-trivial computer vision problem, given that you have to figure out the distance to the "background", especially without stereo vision. (And you'd, at least in the general case, have to make zones volumes rather than areas for this to work.)

That being, quite obviously, out of scope for a first or second iteration of this feature, what would be a much simpler and still useful solution, given that you're already handling presets, is if it were possible to provide per-preset zone definitions.

[NickM-27]

Yeah I've given it some thought but for this first PR I think what I have now is a good starting point. Once it's merged I'll look into how zones to preset mapping could work.

[jhonathas]

This PR will enter?

[72f393f4a1b9dfecdd32641633e9e896]

Hi there great work so far! I run my ip cam through a vpn and proxy feeds to another subnet (docker). Would you be able to add a feature which forces onvif to connect/control using another IP regardless of what the camera responds with? For example I have the proxy at 10.0.0.5 and the camera is 192.168.0.5, so onvif responds with the wrong connection info.

 frigate.ptz                    ERROR   : Unable to connect to camera: Meme: Unknown error: HTTPConnectionPool(host='192.168.0.5', port=8999): Max retries exceeded with url: /onvif/media_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb33e799fd0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
 frigate.http                   ERROR   : Exception on /Meme/ptz/info [GET]
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2529, in wsgi_app
      response = self.full_dispatch_request()
    File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1825, in full_dispatch_request
      rv = self.handle_user_exception(e)
    File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1823, in full_dispatch_request
      rv = self.dispatch_request()
    File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1799, in dispatch_request
      return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
    File "/opt/frigate/frigate/http.py", line 1003, in camera_ptz_info
      return jsonify(current_app.onvif.get_camera_info(camera_name))
    File "/opt/frigate/frigate/ptz.py", line 217, in get_camera_info
      "features": self.cams[camera_name]["features"],
  KeyError: 'features'

My setup doesn't respect the host IP. The service is accessible there, but the camera is on another sub-net and frigate is attempting to connect to an IP which it will never be able to access. Perhaps a "force_host" key in the yml

[NickM-27]

I don't think this is something that frigate can control, the only way that would work is if the onvif library itself supported such an option. Frigate inputs whatever you have in your config directly into the onvif library

[72f393f4a1b9dfecdd32641633e9e896]

I understand. I will need to look for an external solution. TYVM

[sshaikh]

My PTZ camera has a "motion tracking" function where it can physically track motion. I doubt it would work with Frigate, so it's disabled. However once Frigate is able to use PTZ controls, could it track motion (or rather, objects) instead?

[NickM-27]

This has been discussed here: https://github.com/blakeblackshear/frigate/discussions/6695

There's a lot that goes into this complicated feature so it's going to take some time.