mr_score

ampligraph.evaluation.mr_score(ranks)

Mean Rank (MR)

The function computes the mean of of a vector of rankings ranks.

It is used in conjunction with the learning to rank evaluation protocol of evaluation.evaluate_performance.

It is formally defined as follows:

\[MR = \frac{1}{|Q|}\sum_{i = 1}^{|Q|}rank_{(s, p, o)_i}\]

where \(Q\) is a set of triples and \((s, p, o)\) is a triple \(\in Q\).

Note

This metric is not robust to outliers. It is usually used in conjunction with MRR metrics.mrr.

Consider the following example. Each of the two positive triples identified by * are ranked against four corruptions each. When scored by an embedding model, the first triple ranks 2nd, and the other triple ranks first. The resulting MR is:

s        p         o            score   rank
Jack   born_in   Ireland        0.789      1
Jack   born_in   Italy          0.753      2  *
Jack   born_in   Germany        0.695      3
Jack   born_in   China          0.456      4
Jack   born_in   Thomas         0.234      5

s        p         o            score   rank
Jack   friend_with   Thomas     0.901      1  *
Jack   friend_with   China      0.345      2
Jack   friend_with   Italy      0.293      3
Jack   friend_with   Ireland    0.201      4
Jack   friend_with   Germany    0.156      5

MR=1.5

Examples

>>> from ampligraph.evaluation import mr_score
>>> ranks= [5, 3, 4, 10, 1]
>>> mr_score(ranks)
4.6