feature-requests
feature-requests copied to clipboard
Improve ESP32 camera frame rates when streaming video to HA
Describe the problem you have/What new integration you would like
Current frame rates with 640x480 of an OV2640 are > 10 FPS when streaming with MJPEG web camera component, but 2.5 FPS when watching the same feed through HA. Rates remain high when both videos are playing, however when the MJPEG component ceases to stream, rates drop to 2.5 FPS.
Similar effect is seen with the OV5640, where the MJPEG streaming rate is 6.5 FPS average, and drops to 1.5 when streaming through HA API.
(These tests were performed using a Freenove device, although other ESP devices were tested with similar results. Freenove needed a custom component with minimal edits compared to the upstream component: https://github.com/Rudd-O/esphome_freenove_camera_component )
Please describe your use case for this integration and alternatives you've tried:
A higher frame rate would enable much more uses of OV cameras with ESP32 devices.
Additional context
No additional context. I do have this code to calculate the frame rate:
camera:
# ...
on_image:
- component.update: camera_frame_rate
globals:
- id: last_update_times
type: unsigned int[$frame_rate_buffer_size]
restore_value: no
sensor:
- platform: template
name: "Camera frame rate"
id: camera_frame_rate
lambda: |-
float now = millis();
unsigned int max_ = $frame_rate_buffer_size;
unsigned int i;
for (i = max_ - 1; i > 0; i--) {
id(last_update_times)[i] = id(last_update_times)[i-1];
}
id(last_update_times)[0] = now;
float avgdiff = 0.0;
float mostrecentmillis = id(last_update_times)[0];
for (i = 1; i < max_; i++) {
if (mostrecentmillis == 0) {
ESP_LOGD("Camera", "No more updates, breaking at %u", i);
break;
}
avgdiff = mostrecentmillis - id(last_update_times)[i];
if ((id(last_update_times)[i-1] - id(last_update_times)[i]) > 9500) {
ESP_LOGD("Camera", "Big diff, breaking at %u", i);
if (i > 1) {
// This is at least value number three from most
// recent to least recent. Undo its use.
i = i - 1;
avgdiff = mostrecentmillis - id(last_update_times)[i];
ESP_LOGD("Camera", "Compensating avgdiff");
}
break;
}
}
if (avgdiff == 0.0) {
return {};
} else {
avgdiff = avgdiff / i;
float rate = 1000.0 / avgdiff;
//if (rate >= 5.0) {
// rate = floor(rate + 0.5);
//} else {
// rate = floor(rate * 10 + 0.5) / 10;
//}
return rate;
}
update_interval: never
entity_category: diagnostic
state_class: measurement
device_class: frequency
filters:
- heartbeat: 5s
- throttle_average: 5s
- delta: 0.25
Don't forget that with HA API you normally have a real time encrypted connection, that may slow down a bit such transfers of bigger data packs.
By that much? I suspect something else beyond CPU execution time might be wrong.