mr_score

ampligraph.evaluation.mr_score(ranks)

Mean Rank (MR)

The function computes the mean of of a vector of rankings ranks.

It can be used in conjunction with the learning to rank evaluation protocol of ampligraph.evaluation.evaluate_performance().

It is formally defined as follows:

\[MR = \frac{1}{|Q|}\sum_{i = 1}^{|Q|}rank_{(s, p, o)_i}\]

where \(Q\) is a set of triples and \((s, p, o)\) is a triple \(\in Q\).

Note

This metric is not robust to outliers. It is usually presented along the more reliable MRR ampligraph.evaluation.mrr_score().

Consider the following example. Each of the two positive triples identified by * are ranked against four corruptions each. When scored by an embedding model, the first triple ranks 2nd, and the other triple ranks first. The resulting MR is:

s        p         o            score   rank
Jack   born_in   Ireland        0.789      1
Jack   born_in   Italy          0.753      2  *
Jack   born_in   Germany        0.695      3
Jack   born_in   China          0.456      4
Jack   born_in   Thomas         0.234      5

s        p         o            score   rank
Jack   friend_with   Thomas     0.901      1  *
Jack   friend_with   China      0.345      2
Jack   friend_with   Italy      0.293      3
Jack   friend_with   Ireland    0.201      4
Jack   friend_with   Germany    0.156      5

MR=1.5

Examples

>>> from ampligraph.evaluation import mr_score
>>> ranks= [5, 3, 4, 10, 1]
>>> mr_score(ranks)
4.6