PDF version of this document

Generalisation

Generalisation of a model defines its ability to generalise to or represent well images of the modeled class both seen (in the training set) and unseen (not in the training set). A model that comprehensively captures the variation in the modeled class of object should be close, i.e. exhibit low distance, to all the images from that class). In practice this means that all the training examples used to construct the model should be close to model distribution sampled by the model geneted synthetic examples. Given the framework defined for evaluation of specificity above, i.e. a large set of synthetic example images sampled from the model { $I_{j}:j=1..m\}$ and a measure of the distance between images $\vert\cdot\vert$, Generalisation $G$ of a model and the standard error on its measurement $\sigma_{G}$ can be defined as follows:

$$G=\frac{1}{n}\begin{array}{c} n\\ \sum\\ i=1\end{array}min_{j}\,\vert I_{i}-I_{j}\vert,$$ (3)

$$\mathbf{\sigma_{G}}=\frac{SD(min_{\, j}\,\vert I_{i}-I_{j}\vert)}{\sqrt{n-1}},$$ (4)

i.e. it is the average distance from each training image to its nearest neighbour in the image set generated by the model. Once again, good models exhibit low values of Generalisation indicating that the modelled class is well-represented by the model.

Figure 2: Hyperspace representation of the model (metric) evaluation approach

[width = 0.95 ]../Graphics/hyperspace_example.png