PTZ support

[Shamshala]

Hi,

since this "little" project :smiling_face_with_three_hearts: moved even to NVR stage I have to ask (I think I'm not the first one but not able to find it). Is there by any chance a possibility/plan for PTZ control implementation? Or probably cartesian system or some sort of a movement vectors of tracked object output to be able to control something else..?

Thanks (I hope I won't get just a plain "No" :rofl: )

[blakeblackshear]

Not currently, but I do have a PTZ camera and I definitely want to incorporate it in the future.

[highaltidude]

Been in surveillance industry for 10+ years. Hard part of this is so many manufacturers have specific drivers for PTZ control. Maybe incorporating ONVIF is easier, even that is hit or miss a lot of the time.

[Shamshala]

@blakeblackshear Perfect!

@highaltidude Yeah, I'm aware of that. The question was meant more about the "output" itself for other incorporation, rather than "PTZ" in the word of meaning used for surveillance. (also because the abbreviation is more common) Paintball/Watergun turret wouldn't probably be able to use native "PTZ" HW :rofl: .

HomeAssistant even have ONVIF implementation so that could be the way to go as well. Although Frigate looks more like a standalone project so being binded to this HA's integration could be a nuisance.

[hunterjm]

HomeAssistant even have ONVIF implementation so that could be the way to go as well. Although Frigate looks more like a standalone project so being binded to this HA's integration could be a nuisance.

I am actually the code owner of the Home Assistant ONVIF integration, and the python library it depends on. I have recently started contributing to this project, and can definitely help with an ONVIF integration that eventually would include PTZ support. I believe discovery & configuration of cameras via the UI would come first on the feature list, however. Once ONVIF is implemented for that, adding PTZ support shouldn't be too much of a leap.

[sotima]

Just an idea: if supporting PTZ, it might be helpfull to also support multiple masks for the detection. I am using PTZ via HomeAssistant, and if I change the position of the camera, my mask which is defined for the default preset is not adequate for the new position.

[mtweten]

I've been dreaming of implementing some sort of PTZ tracking of objects since I got my PTZ camera, and once frigate dropped I thought it would be sweet if a feature like that could be integrated.

[Shamshala]

@sotima Just an idea: if supporting PTZ, it might be helpfull to also support multiple masks for the detection. I am using PTZ via HomeAssistant, and if I change the position of the camera, my mask which is defined for the default preset is not adequate for the new position.

That would probably require more like a dynamic mask of some sort, don't you think? Based on image or only based on position of the PTZ virtually shifting itself with an image. :scream:

[sotima]

@sotima Just an idea: if supporting PTZ, it might be helpfull to also support multiple masks for the detection. I am using PTZ via HomeAssistant, and if I change the position of the camera, my mask which is defined for the default preset is not adequate for the new position.

That would probably require more like a dynamic mask of some sort, don't you think? Based on image or only based on position of the PTZ virtually shifting itself with an image. 😱

That would be the "gold plated solution"! 😉 Because I am just using preset positions, my idea would have been to define an array of masks, and set the correct mask via homeassistant via an automation that handles the PTZ commands for the camera. Right now, I have to change the position by sending the camera the new position via REST or MQTT (2 different cameras), it would be easy to send an MQTT to frigate with the index of the mask to use at the same time.

[markmghali]

Having PTZ control on my Foscam and Reolink E1 Pro, Would be amazing! This project is absolutely awesome keep it up you all!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[lukedukeus]

Is anyone currently working on this? If so, what progress have you made? If nobody else is working on this, I plan on adding a simple control that just calls a script when the ptz control is clicked.

[dougmaitelli]

I don't know much about this subject, but I believe there is a standard called ONVIF that some cameras implement. What about implementing this standard by default so we don't need to have specific implementation for each camera manufacturer?

[lukedukeus]

Makes sense, but If it calls a script, you can configure it to call the onvif service. If your camera uses http for the ptz controls, you can also call that.

[dougmaitelli]

Yes, I mean. Having the controls would be the first part. These would then call events on the backend layer and these events could have different implementations depending on camera manufacturer. Default integration could be ONVIF, but further integrations could be added over time. Scripts could be a "custom" integration where the user would be able to define any kind of action for the trigger of the ptz events, even http calls

[stevecopeza]

Hi @blakeblackshear and @hunterjm, I'm not a coder, but is there any other way I can help realise what you guys are talking about here to facilitate PTZ controls within Frigate ?

[itkfilelor]

Hi @blakeblackshear and @hunterjm, I'm not a coder, but is there any other way I can help realise what you guys are talking about here to facilitate PTZ controls within Frigate ?

The limiting principle here is, as mentioned above, that there is no one size solution for every camera. It should be a relatively simple thing to do (compared to the magic that frigate already is). The biggest issue I can see is time writing the module, and sourcing the api calls for the different cameras. as well as adding a custom option to define your own api for the less popular cameras.

[fouram]

I would even be OK with implementing the solution that the WebRTC Camera custom card (on HACS) did it: There are predefined buttons that can be enabled (there's a pan/tilt D-pad, and a +/- zoom rocker) and you can assign onvif commands to them. Works well with Amcrest. I would like to see the added feature of holding down on a button and delaying the "stop" command from being sent (right now you can only assign a continuous duration, but each click only gives that specific slice of movement time)

[Hukuma1]

I initially implemented PTZ controls using the Frigate Card: https://github.com/dermotduffy/frigate-hass-card

Then I realized for my Amcrest camera, I really just want to cycle a few presets and that's it. No point in clunky up/down left/right actions which moved my camera unpredictably. So I just switched out the buttons for presets instead.

But now I'm left with the issue that obviously my zones are made for preset 1, and won't work in preset 2 (position). Is there a way to possibly save zones on a per coordinate/position level of camera?

[AaronAutomation]

I initially implemented PTZ controls using the Frigate Card: https://github.com/dermotduffy/frigate-hass-card

Then I realized for my Amcrest camera, I really just want to cycle a few presets and that's it. No point in clunky up/down left/right actions which moved my camera unpredictably. So I just switched out the buttons for presets instead.

But now I'm left with the issue that obviously my zones are made for preset 1, and won't work in preset 2 (position). Is there a way to possibly save zones on a per coordinate/position level of camera?

I am in a similar position with you, would love to see some development for dynamic masking based on pre-defined positions

[bbccdd]

I really just want to cycle a few presets and that's it. No point in clunky up/down left/right actions which moved my camera unpredictably. So I just switched out the buttons for presets instead.

Hi @sotima would you mind sharing a bit more (perhaps a code snippet) on how you call the presets please?

[lukaszzyla]

Hi. Just managed ti get frigate working through rtmp with reolink e1 zoom. It was a bit of troubleshooting but is perfect now. I also would be very hapy just to have presets for PTZ and not necessarily the controls (maybe not in first stage). PTZ presets and a way to program them in Frigate (let's say switch presets every 15 min).

[rsnodgrass]

+1 for presets...this solves a LOT of problems without opening up the complexity of lots of PTZ adjustability.

[jonititan]

Would a plugin be useful here? Perhaps something making use of Stone Soup? That can handle object detection with moving cameras.
https://stonesoup.readthedocs.io/en/v0.1b11/auto_demos/Video_Processing.html#sphx-glr-auto-demos-video-processing-py

[NickM-27]

I have a draft PR and demo here: https://github.com/blakeblackshear/frigate/pull/4715

as it says, this won't be in 0.12

[VictorTorrens]

ptz or onvif integration would be great

[jhansche]

PTZ controls aside (whether manual or automated due to detection), is there any reason to think that Frigate would (or would not) be able to "handle" a camera stream that has its image shift outside of Frigate's knowledge? For example, the Wyze Pan Cam v3 has two separate options that will cause it to pan automatically:

• "Pan Scan" - cycles through preset (customizable) waypoints every 10 seconds. The default waypoints are at 90° intervals (NW, NE, SE, SW), but those waypoints can be customized in the app.
• "Track Motion" - this causes the Pan v3 to automatically detection motion and track the source of movement. It's a little spastic, and sometimes will follow the object all the way until an occlusion, such as behind a wall or other obstruction, to the point where ~75% of the image is unusable. But for the most part, it generally works okay.

My question is more about whether this will cause Frigate to go berserk (for example, will the Pan Scan feature cause it to detect motion every 10 seconds, because the camera is panning and therefore the entire frame is a motion event?); and if there are any recommended settings to accommodate that - i.e. some setting to allow the motion and object detectors to "understand" that the frames are simply panning, rather than triggering a motion event that encompasses 100% of the frame?

[NickM-27]

My question is more about whether this will cause Frigate to go berserk (for example, will the Pan Scan feature cause it to detect motion every 10 seconds, because the camera is panning and therefore the entire frame is a motion event?); and if there are any recommended settings to accommodate that - i.e. some setting to allow the motion and object detectors to "understand" that the frames are simply panning, rather than triggering a motion event that encompasses 100% of the frame?

Currently yes it will count as motion, we are hoping to improve this behavior in an upcoming release

[hawkeye217]

I created this PR as a basic implementation of object autotracking - using a PTZ camera to automatically track an object and keep it in the center of the frame as it moves. With ONVIF support in upcoming 0.13, Frigate should be able to take more advantage of your PTZ camera.