Something seems wrong in your implementation

[robertour]

If I do the following

import rbo 
S = [1,2,3,4,5,6,7,8,0,9]; T = [1,2,3,4,5,6,7,8,9,0]
print(pip_rbo.RankingSimilarity(S, T).rbo(p=.9))

I get 0.6465385908999999

but with https://github.com/dlukes/rbo and (https://towardsdatascience.com/rbo-v-s-kendall-tau-to-compare-ranked-lists-of-items-8776c5182899)[https://towardsdatascience.com/rbo-v-s-kendall-tau-to-compare-ranked-lists-of-items-8776c5182899], I am getting 0.9952170310000001 and 0.995458491249349 respectively.

It might be just that your p is inverted, because

 print(rbo.RankingSimilarity(S, T).rbo(p=.1))

would be 0.9999999989. Still not quite yet, because for the value of p = .9, .995.. seems to makes more sense.

[changyaochen]

@robertour Thanks for pointing this out, I'm looking into it.

-- about an hour later -- It seems that the interpretation could be correct? From the original paper, on page 15, right under equation 7, it says:

The parameter p determines how steep the decline in weights is: the smaller p, the more top-weighted the metric is.

So, in the example you gave, with a small p such as 0.1, we do expect a rbo value close to 1, whereas with a large p such as 0.9, the paper says:

On the other hand, as p approaches arbitrarily close to 1, the weights become arbitrarily flat, and the evaluation becomes arbitrarily deep.

This might be problematic in a way, as from rbo(S, T, p=0.999) to rbo(S, T, p=1.), there might be a large jump, which might not be desired? It could be the inherit properly of this metric?