load_nl27k

ampligraph.datasets.datasets.load_nl27k(check_md5hash=False, clean_unseen=True, split_test_into_top_bottom=True, split_threshold=0.1)

Load the NL27K dataset.

NL27K was originally proposed in [CCS+19]. It is a subset of the Never Ending Language Learning (NELL) dataset [MCH+18], which collects data from web pages. Numeric values on triples represent link uncertainty.

NL27K is loaded from file if it exists at the AMPLIGRAPH_DATA_HOME location. If AMPLIGRAPH_DATA_HOME is not set, the default ~/ampligraph_datasets is checked. If the dataset is not found at either location, it is downloaded and placed in AMPLIGRAPH_DATA_HOME or ~/ampligraph_datasets.

It is divided into three splits:

  • train: 149,100 triples

  • valid: 12,274 triples

  • test: 14,026 triples

Each triple in these splits is associated to a numeric value which represents the importance/relevance of the link.

Dataset

Train

Valid

Test

Entities

Relations

NL27K

149,100

12,274

14,026

27,221

405
Parameters:

  • check_md5hash (bool) – If True check the md5hash of the files (default: False).

  • clean_unseen (bool) – If True, filters triples in validation and test sets that include entities not present in the training set.

  • split_test_into_top_bottom (bool) – Splits the test set by numeric values and returns test_top_split and test_bottom_split by splitting based on sorted numeric values and returning top and bottom k% triples, where k is specified by split_threshold argument.

  • split_threshold (float) – Specifies the top and bottom percentage of triples to return.

Returns:

splits – The dataset splits: {‘train’: train, ‘valid’: valid, ‘test’: test, ‘test_topk’: test_topk, ‘test_bottomk’: test_bottomk, ‘train_numeric_values’: train_numeric_values, ‘valid_numeric_values’:valid_numeric_values, ‘test_numeric_values’: test_numeric_values, ‘test_topk_numeric_values’: test_topk_numeric_values, ‘test_bottomk_numeric_values’: test_bottomk_numeric_values}. Each *_numeric_values split contains numeric values associated to the corresponding dataset split and is a ndarray of shape (n). Each dataset split is a ndarray of shape (n,3). The *_topk and *_bottomk splits are only returned when split_test_into_top_bottom=True and contain the triples ordered by highest/lowest numeric edge value associated. These are typically used at evaluation time aiming at observing a model that assigns the highest rank possible to the _topk and the lowest possible to the _bottomk.

Return type:

dict

Example

>>> from ampligraph.datasets import load_nl27k
>>> X = load_nl27k()
>>> X["train"][0]
['concept:company:business_review' 'concept:competeswith' 'concept:company:miami_herald001']
>>> X['train_numeric_values'][0]
[0.859375]