PredictionIO's evaluation module allows you to streamline the process of testing lots of knobs in engine parameters and deploy the best one out of it using statisically sound cross-validation methods.

There are two key components:


It is our evaluation target. During evaluation, in addition to the train and deploy mode we describe in earlier sections, the engine also generates a list of testing data points. These data points are a sequence of Query and Actual Result tuples. Queries are sent to the engine and the engine responds with a Predicted Result, in the same way as how the engine serves a query.


The evaluator joins the sequence of Query, Predicted Result, and Actual Result together and evaluates the quality of the engine. PredictionIO enables you to implement any metric with just a few lines of code.

PredictionIO Evaluation Overview

We will discuss various aspects of evaluation with PredictionIO.