Evaluation

spotlight.evaluation.mrr_score(model, test, train=None)[source]

Compute mean reciprocal rank (MRR) scores. One score is given for every user with interactions in the test set, representing the mean reciprocal rank of all their test items.

Parameters:
Returns:

mrr scores – Array of MRR scores for each user in test.

Return type:

numpy array of shape (num_users,)

spotlight.evaluation.precision_recall_score(model, test, train=None, k=10)[source]

Compute Precision@k and Recall@k scores. One score is given for every user with interactions in the test set, representing the Precision@k and Recall@k of all their test items.

Parameters:
  • model (fitted instance of a recommender model) – The model to evaluate.
  • test (spotlight.interactions.Interactions) – Test interactions.
  • train (spotlight.interactions.Interactions, optional) – Train interactions. If supplied, scores of known interactions will not affect the computed metrics.
  • k (int or array of int,) – The maximum number of predicted items
Returns:

(Precision@k, Recall@k) – A tuple of Precisions@k and Recalls@k for each user in test. If k is a scalar, will return a tuple of vectors. If k is an array, will return a tuple of arrays, where each row corresponds to a user and each column corresponds to a value of k.

Return type:

numpy array of shape (num_users, len(k))

spotlight.evaluation.rmse_score(model, test)[source]

Compute RMSE score for test interactions.

Parameters:
Returns:

rmse_score – The RMSE score.

Return type:

float

spotlight.evaluation.sequence_mrr_score(model, test, exclude_preceding=False)[source]

Compute mean reciprocal rank (MRR) scores. Each sequence in test is split into two parts: the first part, containing all but the last elements, is used to predict the last element.

The reciprocal rank of the last element is returned for each sequence.

Parameters:
  • model (fitted instance of a recommender model) – The model to evaluate.
  • test (spotlight.interactions.SequenceInteractions) – Test interactions.
  • exclude_preceding (boolean, optional) – When true, items already present in the sequence will be excluded from evaluation.
Returns:

mrr scores – Array of MRR scores for each sequence in test.

Return type:

numpy array of shape (num_users,)