Evaluation

spotlight.evaluation.mrr_score(model, test, train=None)[source]

Compute mean reciprocal rank (MRR) scores. One score is given for every user with interactions in the test set, representing the mean reciprocal rank of all their test items.

Parameters
Returns

mrr scores – Array of MRR scores for each user in test.

Return type

numpy array of shape (num_users,)

spotlight.evaluation.precision_recall_score(model, test, train=None, k=10)[source]

Compute Precision@k and Recall@k scores. One score is given for every user with interactions in the test set, representing the Precision@k and Recall@k of all their test items.

Parameters

  • model (fitted instance of a recommender model) – The model to evaluate.

  • test (spotlight.interactions.Interactions) – Test interactions.

  • train (spotlight.interactions.Interactions, optional) – Train interactions. If supplied, scores of known interactions will not affect the computed metrics.

  • k (int or array of int,) – The maximum number of predicted items

Returns

(Precision@k, Recall@k) – A tuple of Precisions@k and Recalls@k for each user in test. If k is a scalar, will return a tuple of vectors. If k is an array, will return a tuple of arrays, where each row corresponds to a user and each column corresponds to a value of k.

Return type

numpy array of shape (num_users, len(k))

spotlight.evaluation.rmse_score(model, test)[source]

Compute RMSE score for test interactions.

Parameters
Returns

rmse_score – The RMSE score.

Return type

float

spotlight.evaluation.sequence_mrr_score(model, test, exclude_preceding=False)[source]

Compute mean reciprocal rank (MRR) scores. Each sequence in test is split into two parts: the first part, containing all but the last elements, is used to predict the last element.

The reciprocal rank of the last element is returned for each sequence.

Parameters

  • model (fitted instance of a recommender model) – The model to evaluate.

  • test (spotlight.interactions.SequenceInteractions) – Test interactions.

  • exclude_preceding (boolean, optional) – When true, items already present in the sequence will be excluded from evaluation.

Returns

mrr scores – Array of MRR scores for each sequence in test.

Return type

numpy array of shape (num_users,)

spotlight.evaluation.sequence_precision_recall_score(model, test, k=10, exclude_preceding=False)[source]

Compute sequence precision and recall scores. Each sequence in test is split into two parts: the first part, containing all but the last k elements, is used to predict the last k elements.

Parameters

  • model (fitted instance of a recommender model) – The model to evaluate.

  • test (spotlight.interactions.SequenceInteractions) – Test interactions.

  • exclude_preceding (boolean, optional) – When true, items already present in the sequence will be excluded from evaluation.

Returns

mrr scores – Array of MRR scores for each sequence in test.

Return type

numpy array of shape (num_users,)