hits_at_n_score

ampligraph.evaluation.hits_at_n_score(ranks, n)

Hits@N

The function computes how many elements of a vector of rankings ranks make it to the top n positions.

It can be used in conjunction with the learning to rank evaluation protocol of ampligraph.evaluation.evaluate_performance().

It is formally defined as follows:

\[Hits@N = \sum_{i = 1}^{|Q|} 1 \, \text{if } rank_{(s, p, o)_i} \leq N\]

where \(Q\) is a set of triples and \((s, p, o)\) is a triple \(\in Q\).

Consider the following example. Each of the two positive triples identified by * are ranked against four corruptions each. When scored by an embedding model, the first triple ranks 2nd, and the other triple ranks first. Hits@1 and Hits@3 are:

s        p         o            score   rank
Jack   born_in   Ireland        0.789      1
Jack   born_in   Italy          0.753      2  *
Jack   born_in   Germany        0.695      3
Jack   born_in   China          0.456      4
Jack   born_in   Thomas         0.234      5

s        p         o            score   rank
Jack   friend_with   Thomas     0.901      1  *
Jack   friend_with   China      0.345      2
Jack   friend_with   Italy      0.293      3
Jack   friend_with   Ireland    0.201      4
Jack   friend_with   Germany    0.156      5

Hits@3=1.0
Hits@1=0.5
Parameters:
  • ranks (ndarray or list, shape [n] or [n,2]) – Input ranks of n test statements.
  • n (int) – The maximum rank considered to accept a positive.
Returns:

hits_n_score – The Hits@n score

Return type:

float

Examples

>>> import numpy as np
>>> from ampligraph.evaluation.metrics import hits_at_n_score
>>> rankings = np.array([1, 12, 6, 2])
>>> hits_at_n_score(rankings, n=3)
0.5