|
  
Next:Metadata Search on the Up: Model-Based Mining of Previous:Recursive Construction of Models
Model Validation In this section, we describe the mechanism for validating a model [2]. Validation involves executing the
hypothetical model as search or query primitives. Using the HPS as an example, the validation phase includes the comparison between the hypothetical model (in this case, a composite object consisting of
houses surrounded by bushes, plus a weather pattern where a wet summer is followed by a dry summer) and the location of the actual occurrences of the HPS. The relevance of the returned results is
indicated by its similarity measure. Specifically, the performance of the model with respect to the ground truth is measured in terms of precision and recall, defined below. Let X be a template,  be the number of ground truth matches in the database,  be the requested number of results. The query returns  of the matches, where  . In the following definitions, let be fixed, and let  denote the expectation with respect to X.
- Precision,
: This is the proportion of the retrieved results that are relevant. For each template X, define  . Then,
-
- Recall,
: This is the proportion of the relevant results that are retrieved. For each template X let  , the proportion of correct results in a retrieved set of size n. Then ,
-
Both  and  are estimated in the experiments by sample averages, and precision versus recall plots for each template X are obtained by varying n
outside the range  where  . A good fit between the model prediction and ground truth produces precision and recall values are
close to one, indicating a model that is both accurate and efficient. In the rest of this section, we describe strategies for validating a model by searching at multiple
abstraction levels. The discussion is primarily focused on image objects; extension to time series or 3D objects is straightforward.
Next:Metadata Search on the Up: Model-Based Mining of Previous:Recursive Construction of Models |
