mr_score¶
-
ampligraph.evaluation.
mr_score
(ranks)¶ Mean Rank (MR)
The function computes the mean of of a vector of rankings
ranks
.It is used in conjunction with the learning to rank evaluation protocol of
evaluation.evaluate_performance
.It is formally defined as follows:
\[MR = \frac{1}{|Q|}\sum_{i = 1}^{|Q|}rank_{(s, p, o)_i}\]where \(Q\) is a set of triples and \((s, p, o)\) is a triple \(\in Q\).
Note
This metric is not robust to outliers. It is usually used in conjunction with MRR
metrics.mrr
.Consider the following example. Each of the two positive triples identified by
*
are ranked against four corruptions each. When scored by an embedding model, the first triple ranks 2nd, and the other triple ranks first. The resulting MR is:s p o score rank Jack born_in Ireland 0.789 1 Jack born_in Italy 0.753 2 * Jack born_in Germany 0.695 3 Jack born_in China 0.456 4 Jack born_in Thomas 0.234 5 s p o score rank Jack friend_with Thomas 0.901 1 * Jack friend_with China 0.345 2 Jack friend_with Italy 0.293 3 Jack friend_with Ireland 0.201 4 Jack friend_with Germany 0.156 5 MR=1.5
Examples
>>> from ampligraph.evaluation import mr_score >>> ranks= [5, 3, 4, 10, 1] >>> mr_score(ranks) 4.6