In this section, we describe the mechanism for
validating a model [2].
Validation involves
executing the hypothetical model as search or query primitives.
Using the HPS as an example, the validation phase
includes the comparison between the hypothetical model
(in this case, a composite object consisting of
houses surrounded by bushes, plus a weather
pattern where a wet summer is followed by a dry summer)
and the location of the actual occurrences of the HPS.
The relevance of the returned results is indicated by its similarity measure.
Specifically, the performance of the model
with respect to the ground truth is measured in terms of
precision and recall, defined below.
Let X be a template, 
be the number of ground truth matches
in the database, 
be the requested number of results.
The query returns 
of the matches,
where 
.
In the following definitions, let be fixed, and let
denote the expectation with
respect to X.
:
This is the proportion of
the retrieved results that are relevant.
For each template X, define
.
Then,
:
This is the proportion of the
relevant results that are retrieved.
For each template X let 
,
the proportion of correct results in a retrieved
set of size n. Then ,
Both 
and 
are estimated
in the experiments by sample averages, and
precision versus recall plots for each template
X are obtained by varying n outside the
range 
where 
.
A good fit between the model prediction and ground truth produces precision and recall values are close to one, indicating a model that is both accurate and efficient.
In the rest of this section, we describe strategies for validating a model by searching at multiple abstraction levels. The discussion is primarily focused on image objects; extension to time series or 3D objects is straightforward.
| Project home| Technical agenda| Publications| Contact|
