Changing frame rate

[thetable]

Is it possible to change the frame rate at which a video is processed?

I'm writing custom color matchers, some of which are somewhat computationally expensive, and for my purposes, it would be enough to process 1 or 2 frames per second.

[lukemunn]

@thetable Yes would also be really interested in this. Treating a webcam at 30FPS is really hard on the processor. I've tweaked my local version of tracking js to use the 'constraints' parameter for getUserMedia. This establishes ideal and maximum frame rates. The problem is that the on 'track' Event still runs 30 times a second. I can't see where to set this. I would think it should use the camera's (now slow) frame rate.

BTW have also changed to mediaDevices.getUserMedia as Mozilla Dev docs say that navigation.getUserMedia will soon be deprecated.....

tracking.initUserMedia_ = function(element, opt_options) {
  	
  	var constraints = { 
  		video: { 
  			frameRate: { 
  				ideal: 1, 
  				max: 1 
  			} 
  		}, 
  		audio: false 
  	};
  	
	   window.navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {
	  /* use the stream */
	  try {
          element.src = window.URL.createObjectURL(stream);
        } catch (err) {
          element.src = stream;
        }
	}).catch(function(err) {
	  /* handle the error */
	});
  };

[Stan92]

Hi guys.. Did you find a solution? I use the library with a Raspberry PI 3 and the preview is a bit slow

[lukemunn]

Not on my end. :-) Still interested in checking as often as the camera frame rate ticks over (e.g. 10FPS, 15FPS, etc).

[jamiesarahg]

Has anyone found a way to only track after a certain number of frames? I want to reduce the computational weight of the tracking by looking less frequently

[maniacally]

I'm throttling the 'track' event handler but it's still killing the cpu. Would love a proper interval-based solution to this.

[maniacally]

@jamiesarahg I implemented this in my local version. If you pull down the tracking.js file from the build folder and replace the tracking.trackVideo_ function with the following, you'll then be able to frame limit by passing in the number of frames per second like this:

tracking.track('#my-video', tracker, { camera: true, fps:2 });

here's the new version of trackVideo_:

tracking.trackVideo_ = function(element, tracker, options) {
    var canvas = document.createElement('canvas');
    var context = canvas.getContext('2d');
    var width;
    var height;

    options = options || {}
    options.fps = options.fps || 30

    var fpsInterval, now, then, elapsed;

    var resizeCanvas_ = function() {
      width = element.offsetWidth;
      height = element.offsetHeight;
      canvas.width = width;
      canvas.height = height;
    };
    resizeCanvas_();
    element.addEventListener('resize', resizeCanvas_);

    var requestId;
    var requestAnimationFrame_ = function() {
      requestId = window.requestAnimationFrame(function() {
        requestAnimationFrame_();

        // calc elapsed time since last loop
        now = Date.now();
        elapsed = now - then;

        // if enough time has elapsed, draw the next frame
        if (elapsed > fpsInterval) {
          console.log('running frame')       
          // Get ready for next frame by setting then=now, but also adjust for your
          // specified fpsInterval not being a multiple of RAF's interval (16.7ms)
          then = now - (elapsed % fpsInterval);

          if (element.readyState === element.HAVE_ENOUGH_DATA) {
            try {
              // Firefox v~30.0 gets confused with the video readyState firing an
              // erroneous HAVE_ENOUGH_DATA just before HAVE_CURRENT_DATA state,
              // hence keep trying to read it until resolved.
              context.drawImage(element, 0, 0, width, height);
            } catch (err) {}
            tracking.trackCanvasInternal_(canvas, tracker);
          } 
        }
        else
        console.log('skipping frame')       
      });
    };

    var task = new tracking.TrackerTask(tracker);
    task.on('stop', function() {
      window.cancelAnimationFrame(requestId);
    });
    task.on('run', function() {
      fpsInterval = 1000 / options.fps;
      then = Date.now();
      requestAnimationFrame_();
    });
    return task.run();
  };

[tetreault]

Chiming in here as I built a project using trackingjs for eye detection on video and images. on desktops the video was fine on my 2017 MBP but was running horribly on peoples laptops that had CPUs < 2Ghz.

Inside of tracking.trackVideo I just slightly modified the resizeCanvas_ function to just set the width and height of the internal canvas by half and that significantly improved the experience on these people's laptops that were 3+ years old.

    var resizeCanvas_ = function() {
      width = element.offsetWidth;
      height = element.offsetHeight;
      canvas.width = width / 2; 
      canvas.height = height / 2;
    };

[Jylomaki]

I used an alternative solution, by by-passing the tracking.track method, instead calling directly the tracker.track method from the tracker. And you just use any method you want to control the frame track rate in your video handling thread.

Though this method need giving imgData, width, height as parameter (gotten from ctx.drawImage), but I think it allows using webworkers as it disconnect from the video element.

I didn't find anything related to using previous frames in tracking so I think it's okay to short-circuit , but tell me if I'm mistaken about this part. Also as ctx.drawImage was already called internally it doesn't seem like this method duplicate method calls.