Feature - Upload speed calculator

[colevoss]

What?

While using s3.FineUploaderBasic and wanting to know the speed at which my uploads are transferring, I have write this functionality in and around my FineUploader instances. I believe this would be a fantastic feature to have ready to use in the API that would enhance the standard FineUploader interface or any custom interfaces build on top of it.

How?

The way I have been calculating the speed of file transfers has been by taking samples of an Uploader record that stores the totalBytes and uploadedBytes given to the onTotalProgress callback.

From these samples I can create a moving average of the amount of bytes uploaded per interval. I keep 5 samples in my sample queue at a time and pop old samples off while pushing new ones on. Then I just average that list of samples. I have also read about weighted moving averages where the more stale a sample is, the less it contributes to the average but just a standard moving average has worked fine in the past.

When?

Whenever anyone (myself included) thinks this is a good idea and has time.

Thanks

[pafro]

+1 for this.

@colevoss Any chance of seeing your calculation code and how it's implemented - this feature is on my UI list and I really like not 'reinventing the wheel!'

Is your code looking at a each uploading file and also the 'overall' upload speed (all files combined?)

[rnicholus]

Unlikely that this will be part of the API, but I can see a blog post that explains how to calculate current or average transfer rate.

[rmckeel]

Hello, yes, this is an important feature for 'big uploads' that can take hours or days. I have code that I use for this problem. When a user pauses uploading, it clears out the array of samples. Perhaps this will be good fodder for the blog entry (correcting/improving as you see fit)? :)

Ryan

.on("totalProgress", function(event, totalUploadedBytes, totalBytes) {
        var progressPercent = (totalUploadedBytes / totalBytes).toFixed(2);
        if(isNaN(progressPercent) || progressPercent >= 1) {
          $('#progress-text').text('');
          if($progressBarContainer) $progressBarContainer.slideUp();
        } else {
          var progress = (progressPercent * 100).toFixed() + '%';

          $('#progress-text').text(progress);
          if($progressBarContainer) $progressBarContainer.slideDown();
        }

        // upload speed
        uploadSpeeds.push({
          totalUploadedBytes: totalUploadedBytes,
          currentTime: new Date().getTime()
        });
        var minSamples = 6;
        var maxSamples = 20;
        if(uploadSpeeds.length > maxSamples) {
          // remove first element
          uploadSpeeds.shift();
        }
        if(uploadSpeeds.length >= minSamples) {
          try {
            var firstSample = uploadSpeeds[0];
            var lastSample = uploadSpeeds[uploadSpeeds.length - 1];
            var progressBytes = lastSample.totalUploadedBytes - firstSample.totalUploadedBytes;
            var progressTimeMS = lastSample.currentTime - firstSample.currentTime;
            // megabytes per second
            //var MBps = (progressBytes / (progressTimeMS / 1000) / (1024 * 1024)).toFixed(2);
            // megabits per second
            var Mbps = ((progressBytes * 8) / (progressTimeMS / 1000) / (1000 * 1000)).toFixed(2);
            //console.log(progressBytes, progressTimeMS, Mbps, uploadSpeeds.length);
            if(Mbps > 0) {
              $('#uploader-speed').text(Mbps + ' Mbps');
            } else {
              $('#uploader-speed').text('');
            }
          } catch (err) {

          }
        }
      })

[rnicholus]

Thanks for the code @rmckeel. I don't have a specific date for this blog post, as there are a number of other items ahead of this in my queue. But I don't expect it to be very difficult to throw something together once I am able to do so.

[ghost]

I additionally need to know the remaining time of the upload(s). Any idea? @rnicholus The UI code / documentation should contain an example for this.

[rnicholus]

@JHGitty Please do submit a pull request against the docs with a solution that works for you. I am also considering opening up the wiki so these sorts of things are easier to contribute.

[ghost]

@rnicholus Great idea and thanks! I think this is helpful: http://stackoverflow.com/questions/21162749/how-do-i-calcuate-the-time-remaining-for-my-upload

[ghost]

I use JQuery. Here is my code. This code only works out-of-the-box if you have only 1 uploader box on your page. Otherwise you need to change the hardcoded div #progress-text and #uploader-speed.

• Add remaining time
• Add total progress %
• Add upload speed

            var uploadSpeeds = [];

            function updateSpeedText(totalUploadedBytes, totalBytes) {
                var progressPercent = (totalUploadedBytes / totalBytes).toFixed(2);
                if (isNaN(progressPercent) || progressPercent >= 1) {
                    $('#progress-text').text('');
                } else {
                    var progress = (progressPercent * 100).toFixed() + '% total';
                    $('#progress-text').text(progress);
                }

                // upload speed
                uploadSpeeds.push({
                    totalUploadedBytes: totalUploadedBytes,
                    currentTime: new Date().getTime()
                });
                var minSamples = 6;
                var maxSamples = 20;
                if (uploadSpeeds.length > maxSamples) {
                    // remove first element
                    uploadSpeeds.shift();
                }
                if (uploadSpeeds.length >= minSamples) {
                    try {
                        var firstSample = uploadSpeeds[0];
                        var lastSample = uploadSpeeds[uploadSpeeds.length - 1];
                        var progressBytes = lastSample.totalUploadedBytes - firstSample.totalUploadedBytes;
                        var progressTimeMS = lastSample.currentTime - firstSample.currentTime;
                        // megabytes per second
                        //var MBps = (progressBytes / (progressTimeMS / 1000) / (1024 * 1024)).toFixed(2);
                        // megabits per second
                        var Mbps = ((progressBytes * 8) / (progressTimeMS / 1000) / (1000 * 1000)).toFixed(2);
                        //console.log(progressBytes, progressTimeMS, Mbps, uploadSpeeds.length);
                        if (Mbps > 0) {
                            $('#uploader-speed').text(Mbps + ' Mbps / remaining: ' + ((totalBytes - totalUploadedBytes) / progressBytes).toFixed(1) + ' seconds');
                        } else {
                            $('#uploader-speed').text('');
                        }
                    } catch (err) {

                    }
                }
            }
        });

Don't forget to add a callback to FineUploader:

                callbacks: {
                    onProgress: function (id,name,totalUploadedBytes, totalBytes) {
                        updateSpeedText(totalUploadedBytes, totalBytes);
                    }
                }

And don't forget to add the 2 divs to your template:

                <div id="uploader-speed"></div>
                <div id="progress-text"></div>

(I have added them inner div qq-total-progress-bar-container)

[rnicholus]

I think it would be great to publish something like this as a small library, published to npm. This follows the lodash model, which I appreciate. Then, Fine Uploader users could pull it into their project and easily integrate it for upload speed reporting. I don't see any need to use jQuery here though, or any dependencies for that matter.