Evaluation¶

spotlight.evaluation.
mrr_score
(model, test, train=None)[source]¶ Compute mean reciprocal rank (MRR) scores. One score is given for every user with interactions in the test set, representing the mean reciprocal rank of all their test items.
Parameters:  model (fitted instance of a recommender model) – The model to evaluate.
 test (
spotlight.interactions.Interactions
) – Test interactions.  train (
spotlight.interactions.Interactions
, optional) – Train interactions. If supplied, scores of known interactions will be set to very low values and so not affect the MRR.
Returns: mrr scores – Array of MRR scores for each user in test.
Return type: numpy array of shape (num_users,)

spotlight.evaluation.
precision_recall_score
(model, test, train=None, k=10)[source]¶ Compute Precision@k and Recall@k scores. One score is given for every user with interactions in the test set, representing the Precision@k and Recall@k of all their test items.
Parameters:  model (fitted instance of a recommender model) – The model to evaluate.
 test (
spotlight.interactions.Interactions
) – Test interactions.  train (
spotlight.interactions.Interactions
, optional) – Train interactions. If supplied, scores of known interactions will not affect the computed metrics.  k (int or array of int,) – The maximum number of predicted items
Returns: (Precision@k, Recall@k) – A tuple of Precisions@k and Recalls@k for each user in test. If k is a scalar, will return a tuple of vectors. If k is an array, will return a tuple of arrays, where each row corresponds to a user and each column corresponds to a value of k.
Return type: numpy array of shape (num_users, len(k))

spotlight.evaluation.
rmse_score
(model, test)[source]¶ Compute RMSE score for test interactions.
Parameters:  model (fitted instance of a recommender model) – The model to evaluate.
 test (
spotlight.interactions.Interactions
) – Test interactions.
Returns: rmse_score – The RMSE score.
Return type: float

spotlight.evaluation.
sequence_mrr_score
(model, test, exclude_preceding=False)[source]¶ Compute mean reciprocal rank (MRR) scores. Each sequence in test is split into two parts: the first part, containing all but the last elements, is used to predict the last element.
The reciprocal rank of the last element is returned for each sequence.
Parameters:  model (fitted instance of a recommender model) – The model to evaluate.
 test (
spotlight.interactions.SequenceInteractions
) – Test interactions.  exclude_preceding (boolean, optional) – When true, items already present in the sequence will be excluded from evaluation.
Returns: mrr scores – Array of MRR scores for each sequence in test.
Return type: numpy array of shape (num_users,)

spotlight.evaluation.
sequence_precision_recall_score
(model, test, k=10, exclude_preceding=False)[source]¶ Compute sequence precision and recall scores. Each sequence in test is split into two parts: the first part, containing all but the last k elements, is used to predict the last k elements.
Parameters:  model (fitted instance of a recommender model) – The model to evaluate.
 test (
spotlight.interactions.SequenceInteractions
) – Test interactions.  exclude_preceding (boolean, optional) – When true, items already present in the sequence will be excluded from evaluation.
Returns: mrr scores – Array of MRR scores for each sequence in test.
Return type: numpy array of shape (num_users,)