View source

The <tt>Heuristic</tt> class is declared in [[API: Heuristic class|<tt>mash/heuristic.h</tt>]].

To implement a new heuristic, you'll declare a new class that inherit from <tt>Heuristic</tt>. See the [[heuristics development]] page for detailed usage examples and information about the SDK.


__TOC__


== What is a heuristic? ==

A '''heuristic''' in the MASH project is an image feature extractor. Its job is to compute some numerical quantities, called '''features''', given an image, the coordinates of a '''point''' in this image and the extent of a '''region of interest''' around that point:

[[Image:HeuristicFlow.png|center|link=]]


== The features ==

A feature is a '''floating-point number'''. Its meaning and value is heuristic-dependent: the MASH system makes no assumption about it. However, the overall purpose of features is to make the life of machine learning easier. This is generally achieved by including invariance in the heuristic, such that when the image change in a certain way (e.g. illumination, translation, rotation, etc.) that should not matter for the system, the computed features do not change.

== Dimension of a heuristic ==

All the features computed by a heuristic are grouped in a '''features vector'''. The size of this features vector is called the '''dimension''' of the heuristic. This dimension can only depend on the size of the '''region of interest''' (in other words: as long as the size of the region of interest is the same, the dimension of the heuristic is the same for any point in any image).


== Region of interest ==

A '''region of interest''' is a '''square''' area, centered on a '''point'''. It is defined by its '''extent''' and the '''coordinates''' of the point. Its dimensions (width and height) are given by: <math>extent*2+1</math> (in pixels).

[[Image:ROI.png|center|link=]]


== Life-cycle of a heuristic ==

The methods that you must implement when writing a heuristic will always be called over the course of an experiment in the order shown below.

Sequences of images are relevant only for goal-planning. In that case there is a notion of time, and heuristics may implement some form of memory. For image classification and object detection, this loop can be ignored.

The visiting order of the loops is totally undefined. The same image or location may be visited several times, while others may not be visited at all.

Finally, the method <tt>dim()</tt> may be called at any moment over the life of the heuristic.

: <tt>init()</tt>
: '''Loop over sequences of images'''
:: <tt>prepareForSequence()</tt>
:: '''Loop over some images in the sequence'''
::: <tt>prepareForImage()</tt>
::: '''Loop over some locations in the image'''
:::: <tt>prepareForCoordinates()</tt>
:::: '''Loop over some feature indexes'''
::::: <tt>computeFeature()</tt>
:::: '''End loop over feature indexes'''
:::: <tt>finishForCoordinates()</tt>
::: '''End loop over locations'''
::: <tt>finishForImage()</tt>
:: '''End loop over images'''
:: <tt>finishForSequence()</tt>
: '''End loop over sequences'''
: <tt>terminate()</tt>

== Attributes of the class ==

The <tt>Heuristic</tt> class has the following attributes, that you must use to implement your own heuristic:

<code cpp>
class Heuristic
{
    ....

    unsigned int    nb_views;       // Total number of views
    unsigned int    roi_extent;     // Extent of the region of interest currently processed
    coordinates_t   coordinates;    // Coordinates of the point currently processed
    Image*          image;          // The image currently processed
};
</code>

With <tt>coordinates_t</tt> defined as follows:
<code cpp>
struct coordinates_t
{
    unsigned int x; // X position in pixels
    unsigned int y; // Y position in pixels
};
</code>


== Required methods ==

The following methods '''must''' be implemented by all the heuristics:
<code cpp>
class Heuristic
{
    ....

public:
    //----------------------------------------------------------------------
    /// @brief  Returns the number of features this heuristic computes
    ///
    /// When this method is called, the following attributes are initialized:
    ///     - nb_views
    ///     - roi_extent
    //----------------------------------------------------------------------
    virtual unsigned int dim() = 0;


    //----------------------------------------------------------------------
    /// @brief  Computes the specified feature
    ///
    /// When this method is called, the following attributes are initialized:
    ///     - nb_views
    ///     - roi_extent
    ///     - image
    ///     - coordinates
    //----------------------------------------------------------------------
    virtual scalar_t computeFeature(unsigned int feature_index) = 0;

    ....
};
</code>


== Optional methods ==

The following methods '''can''' be implemented by the heuristics that needs them:
<code cpp>
class Heuristic
{
    ....

public:
    //----------------------------------------------------------------------
    /// @brief  Called once at the creation of the heuristic
    ///
    /// Pre-computes all the data that will never change during the
    /// life-time of the heuristic
    ///
    /// When this method is called, the following attributes are initialized:
    ///     - nb_views
    ///     - roi_extent
    ///
    /// @remark The implementation of this method is optional
    //----------------------------------------------------------------------
    virtual void init() {}

    //----------------------------------------------------------------------
    /// @brief  Called once when the heuristic is destroyed
    ///
    /// Frees the memory allocated by the init() method
    ///
    /// @remark This method must be implemented if init() is used and
    ///         allocated some memory
    //----------------------------------------------------------------------
    virtual void terminate() {}

    //----------------------------------------------------------------------
    /// @brief  Called once per sequence, before any computation 
    ///
    /// Allocates and initializes the data the heuristic will need to
    /// compute features on a stream of images (like a video)
    ///
    /// When this method is called, the following attributes are initialized:
    ///     - nb_views
    ///     - roi_extent
    ///
    /// @remark The implementation of this method is optional
    //----------------------------------------------------------------------
    virtual void prepareForSequence() {}

    //----------------------------------------------------------------------
    /// @brief  Called once per sequence, after any computation 
    ///
    /// Frees the memory allocated by the prepareForSequence() method
    ///
    /// @remark This method must be implemented if prepareForSequence() is
    ///         used and allocated some memory
    //----------------------------------------------------------------------
    virtual void finishForSequence() {}

    //----------------------------------------------------------------------
    /// @brief  Called once per image, before any computation 
    ///
    /// Pre-computes from a full image the data the heuristic will need to
    /// compute features at any coordinates in the image
    ///
    /// When this method is called, the following attributes are initialized:
    ///     - nb_views
    ///     - roi_extent
    ///     - image
    ///
    /// @remark The implementation of this method is optional
    //----------------------------------------------------------------------
    virtual void prepareForImage() {}

    //----------------------------------------------------------------------
    /// @brief  Called once per image, after any computation 
    ///
    /// Frees the memory allocated by the prepareForImage() method
    ///
    /// @remark This method must be implemented if prepareForImage() is used
    ///         and allocated some memory
    //----------------------------------------------------------------------
    virtual void finishForImage() {}

    //----------------------------------------------------------------------
    /// @brief  Called once per coordinates, before any computation
    ///
    /// Pre-computes the data the heuristic will need to compute features
    /// at the given coordinates
    ///
    /// When this method is called, the following attributes are initialized:
    ///     - nb_views
    ///     - roi_extent
    ///     - image
    ///     - coordinates
    ///
    /// @remark The implementation of this method is optional
    //----------------------------------------------------------------------
    virtual void prepareForCoordinates() {}

    //----------------------------------------------------------------------
    /// @brief  Called once per coordinates, after any computation 
    ///
    /// Frees the memory allocated by the prepareForCoordinates() method
    ///
    /// @remark This method must be implemented if prepareForCoordinates()
    ///         is used and allocated some memory
    //----------------------------------------------------------------------
    virtual void finishForCoordinates() {}

    ....
};
</code>