facets icon indicating copy to clipboard operation
facets copied to clipboard

High percentile value raises error

Open schlick opened this issue 4 years ago • 2 comments

Using Math.percentile sometimes fails for a high percentile. Example:

require 'facets/math/percentile'
Math.percentile([1,2,3], 90)
NoMethodError: undefined method `-' for nil:NilClass
	from /Users/michael/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/facets-3.1.0/lib/standard/facets/math/percentile.rb:33:in `percentile'
	from (irb):4
	from /Users/michael/.rbenv/versions/2.3.6/bin/irb:11:in `<main>'

schlick avatar Aug 14 '19 07:08 schlick

At L29:

s1 = sorted_array[whole]

whole is 3 in the provided example and so returns nil for s1. Then L33 tries to take s0 from nil.

schlick avatar Aug 14 '19 07:08 schlick

@schlick and all future people who encounter this problem:

The implementation of percentile in facets appears to only account for condition 1 in the description of "Estimation of percentiles" at the NIST link which is "For 0<k<N"

I encountered the same bug you did, which I think is more frequently encountered with small arrays at high percentiles, K is often >= N which is condition 3. In that case the algorithm should just return sorted_array.last

I have no idea how to contribute to this repo or if it is still being maintained, but the below code is I think a full implementation of the NIST algorithm for calculating percentiles. What was very helpful for me was seeing the worked example of this method of calculation on the wikipedia page.

def get_percentile(array, percentile)
  array = array.sort
  percentile = percentile.to_f / 100
  rank = percentile * (array.length + 1)
  if rank >= array.length
    array.last
  elsif rank.truncate == 0
    array.first
  else
    k = rank.truncate
    d = rank % 1
    array[k-1] + (d * (array[k] - array[k - 1]))
  end
end

savfischer avatar Oct 21 '20 10:10 savfischer