mr_score¶
-
ampligraph.evaluation.
mr_score
(ranks)¶ Mean Rank (MR)
The function computes the mean of of a vector of rankings
ranks
.It can be used in conjunction with the learning to rank evaluation protocol of
ampligraph.evaluation.evaluate_performance()
.It is formally defined as follows:
\[MR = \frac{1}{|Q|}\sum_{i = 1}^{|Q|}rank_{(s, p, o)_i}\]where \(Q\) is a set of triples and \((s, p, o)\) is a triple \(\in Q\).
Note
This metric is not robust to outliers. It is usually presented along the more reliable MRR
ampligraph.evaluation.mrr_score()
.Consider the following example. Each of the two positive triples identified by
*
are ranked against four corruptions each. When scored by an embedding model, the first triple ranks 2nd, and the other triple ranks first. The resulting MR is:s p o score rank Jack born_in Ireland 0.789 1 Jack born_in Italy 0.753 2 * Jack born_in Germany 0.695 3 Jack born_in China 0.456 4 Jack born_in Thomas 0.234 5 s p o score rank Jack friend_with Thomas 0.901 1 * Jack friend_with China 0.345 2 Jack friend_with Italy 0.293 3 Jack friend_with Ireland 0.201 4 Jack friend_with Germany 0.156 5 MR=1.5
Parameters: ranks (ndarray or list, shape [n] or [n,2]) – Input ranks of n test statements. Returns: mr_score – The MR score Return type: float Examples
>>> from ampligraph.evaluation import mr_score >>> ranks= [5, 3, 4, 10, 1] >>> mr_score(ranks) 4.6