Tracking in OpenTLD aka Predator

As my Google Summer of Code 2012 project, I have to port OpenTLD to python using OpenCV and SimpleCV. OpenTLD a.k.a. Predator was first made by Zdenek Kalal in MATLAB. OpenTLD. It is one of the most reliable algorithms to track objects. The algorithm include on-line training and learning.
The algorithm which consists of off-line training take days and lots of data to train and learn.

Here’s my brief understanding of how OpenTLD algorithm works.

As the name suggests, it consists of three main parts.

  • Tracking
  • Learning
  • Detection



Adaptive Tracking is used in OpenTLD. A Median Flow Tracker is made using Lucas-Kanede Tracker with pyramids and with the help of Forward-Backward error, and focusing on 50% of the most reliable points.
As Zdenek Kalal quoted in his GoogleTech Talk about Predator

Every tracker eventually fails and requires a detector.”


Classifiers are continuously trained from each and every frame. For every frame, Classifiers are evaluated. Errors are estimated via feedback. According to the feedback, classifiers are updated to detect more efficiently.
Ensemble classifier and 1NN classifier are used in detection.

feedback in openTLD

The tracker learns using P-N learning (Positive-Negative) which learns an object classifier and labels all the patches as “object”(positive) and “background”(negative).It uses a tracker for providing positive and detector for negative training examples.

PN Learning - OpenTLD

I have started working on OpenTLD for couple of weeks now. I have made a Median Flow tracker for “Tracking” part of OpenTLD. Here’s how the “Tracking” part works in OpenTLD.

This is how Median Flow Tracker works:

  • Initialize points to a grid

  • Get Filled points in the Bounding Box

  • Track points between frame

  • Points are tracked using Lucas-Kanede Tracker with pyramids.

  • Estimate reliability of the points

  • To get reliable points, Forward-Backward error method is used. In FB method, points are tracked twice.
    tracked points for current image -> previous image
    tracked points for previous image -> current image
    So, intersection of both of point sets would give me reliable tracked points.
    Forward-Backward Error

  • Filter out 50% of the outliers

  • 50% of the points are filtered out using median filter. First the median is calculated for the vector of points, and most reliable points are chosen.

  • Estimate the new bounding box

  • New bounding box is estimated based on all relative distance changes of all points to every point. The median of the relative value is used for calculation. Predict Bounding Box

MF Tracker GitHub


P.S. Now working on Learning part.

One thought on “Tracking in OpenTLD aka Predator

  1. The ALIEN Visual Tracker application IS OUT!
    Download it here:
    (available for Windows7 64bit).

    The ALIEN visual tracker is a generic visual object tracker achieving state of the art performance. The object is selected at run-time by drawing a bounding box around it and then its appearance is learned and tracked as time progresses. The ALIEN tracker has been shown to outperform other competitive trackers, especially in the case of long-term tracking, large amount of camera blur, low frame rate videos and severe occlusions including full object disappearance.

    The scientific paper introducing the technology behind the tracker will appear at the 12th European Conference in Computer Vision 2012 under the following title:
    β€’ FaceHugger: The ALIEN Tracker Applied to Faces. In Proceedings of European Conference on Computer Vision (ECCV) – DEMO Session — 2012 Florence Italy.
    A real time demo of the released downloadable application ( will also be given during the conference [1].
    Video demos showing the capability of this novel technology may be seen here

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s