boofcv.alg.tracker.circulant

Class CirculantTracker<T extends boofcv.struct.image.ImageGray<T>>

java.lang.Object

boofcv.alg.tracker.circulant.CirculantTracker<T>

public class CirculantTracker<T extends boofcv.struct.image.ImageGray<T>>
extends java.lang.Object

Tracker that uses the theory of Circulant matrices, Discrete Fourier Transform (DCF), and linear classifiers to track a target and learn its changes in appearance [1]. The target is assumed to be rectangular and has fixed size and location. A dense local search is performed around the most recent target location. The search is done quickly using the DCF.

Tracking is performed using texture information. Since only one description of the target is saved, tracks can drift over time. Tracking performance seems to improve if the object has distinctive edges.

CHANGES FROM PAPER:

• Input image is sampled into a square work region of constant size to improve runtime speed of FFT.
• Peak of response is found using mean-shift. Provides sub-pixel precision.
• Pixels outside the image are assigned random values to avoid the tracker from fitting to them. Ideally they wouldn't be processed, but that is complex to implement

[1] Henriques, Joao F., et al. "Exploiting the circulant structure of tracking-by-detection with kernels." Computer Vision–ECCV 2012. Springer Berlin Heidelberg, 2012. 702-715.

Field Summary

Fields

Modifier and Type	Field and Description
protected boofcv.struct.image.GrayF64	cosine
protected boofcv.struct.image.GrayF64	gaussianWeight
protected boofcv.struct.image.InterleavedF64	gaussianWeightDFT
protected float	offX
protected float	offY
protected georegression.struct.shapes.RectangleLength2D_F32	regionOut
protected georegression.struct.shapes.RectangleLength2D_F32	regionTrack
protected boofcv.struct.image.GrayF64	template
protected boofcv.struct.image.GrayF64	templateNew

Constructor Summary

Constructors

Constructor and Description
CirculantTracker(double output_sigma_factor, double sigma, double lambda, double interp_factor, double padding, int workRegionSize, double maxPixelValue, boofcv.alg.interpolate.InterpolatePixelS<T> interp) Configure tracker

Method Summary

Modifier and Type	Method and Description
static void	circshift(boofcv.struct.image.GrayF64 a, boofcv.struct.image.GrayF64 b)
protected static void	computeAlphas(boofcv.struct.image.InterleavedF64 yf, boofcv.struct.image.InterleavedF64 kf, double lambda, boofcv.struct.image.InterleavedF64 alphaf) new_alphaf = yf ./ (fft2(k) + lambda); %(Eq.
protected static void	computeCosineWindow(boofcv.struct.image.GrayF64 cosine) Computes the cosine window
protected void	computeGaussianWeights(int width) Computes the weights used in the gaussian kernel This isn't actually symmetric for even widths.
void	dense_gauss_kernel(double sigma, boofcv.struct.image.GrayF64 x, boofcv.struct.image.GrayF64 y, boofcv.struct.image.GrayF64 k) Gaussian Kernel with dense sampling.
static void	elementMultConjB(boofcv.struct.image.InterleavedF64 a, boofcv.struct.image.InterleavedF64 b, boofcv.struct.image.InterleavedF64 output) Element-wise multiplication of 'a' and the complex conjugate of 'b'
protected static void	gaussianKernel(double xx, double yy, boofcv.struct.image.GrayF64 xy, double sigma, boofcv.struct.image.GrayF64 output) Computes the output of the Gaussian kernel for each element in the target region k = exp(-1 / sigma^2 * max(0, (xx + yy - 2 * xy) / numel(x)));
protected void	get_subwindow(T image, boofcv.struct.image.GrayF64 output) Copies the target into the output image and applies the cosine window to it.
boofcv.struct.image.GrayF64	getResponse()
georegression.struct.shapes.RectangleLength2D_F32	getTargetLocation() The location of the target in the image
boofcv.struct.image.GrayF64	getTargetTemplate() Visual appearance of the target
static double	imageDotProduct(boofcv.struct.image.GrayF64 a) Computes the dot product of the image with itself
void	initialize(T image, int x0, int y0, int regionWidth, int regionHeight) Initializes tracking around the specified rectangle region
protected void	initialLearning(T image) Learn the target's appearance.
void	performLearning(T image) Update the alphas and the track's appearance
void	performTracking(T image) Search for the track in the image and
protected void	resizeImages(int workRegionSize)
void	setTrackLocation(int x0, int y0, int regionWidth, int regionHeight) Used to change the track's location.
protected void	subpixelPeak(int peakX, int peakY) Refine the local-peak using a search algorithm for sub-pixel accuracy.
protected void	updateTrackLocation(T image) Find the target inside the current image by searching around its last known location

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

templateNew

protected boofcv.struct.image.GrayF64 templateNew

template

protected boofcv.struct.image.GrayF64 template

cosine

protected boofcv.struct.image.GrayF64 cosine

regionTrack

protected georegression.struct.shapes.RectangleLength2D_F32 regionTrack

regionOut

protected georegression.struct.shapes.RectangleLength2D_F32 regionOut

gaussianWeight

protected boofcv.struct.image.GrayF64 gaussianWeight

gaussianWeightDFT

protected boofcv.struct.image.InterleavedF64 gaussianWeightDFT

offX

protected float offX

offY

protected float offY

Constructor Detail

CirculantTracker

public CirculantTracker(double output_sigma_factor,
                        double sigma,
                        double lambda,
                        double interp_factor,
                        double padding,
                        int workRegionSize,
                        double maxPixelValue,
                        boofcv.alg.interpolate.InterpolatePixelS<T> interp)

Configure tracker

Parameters:

output_sigma_factor - spatial bandwidth (proportional to target) Try 1.0/16.0

sigma - Sigma for Gaussian kernel in linear classifier. Try 0.2

lambda - Try 1e-2

interp_factor - Try 0.075

padding - Padding added around the selected target. Try 1

workRegionSize - Size of work region. Best if power of 2. Try 64

maxPixelValue - Maximum pixel value. Typically 255

Method Detail

initialize

public void initialize(T image,
                       int x0,
                       int y0,
                       int regionWidth,
                       int regionHeight)

Initializes tracking around the specified rectangle region

Parameters:

image - Image to start tracking from

x0 - top-left corner of region

y0 - top-left corner of region

regionWidth - region's width

regionHeight - region's height

setTrackLocation

public void setTrackLocation(int x0,
                             int y0,
                             int regionWidth,
                             int regionHeight)

Used to change the track's location. If this method is used it is assumed that tracking is active and that the appearance of the target has not changed

Parameters:

x0 - top-left corner of region

y0 - top-left corner of region

regionWidth - region's width

regionHeight - region's height

initialLearning

protected void initialLearning(T image)

Learn the target's appearance.

computeCosineWindow

protected static void computeCosineWindow(boofcv.struct.image.GrayF64 cosine)

Computes the cosine window

computeGaussianWeights

protected void computeGaussianWeights(int width)

Computes the weights used in the gaussian kernel This isn't actually symmetric for even widths. These weights are used has label in the learning phase. Closer to one the more likely it is the true target. It should be a peak in the image center. If it is not then it will learn an incorrect model.

resizeImages

protected void resizeImages(int workRegionSize)

performTracking

public void performTracking(T image)

Search for the track in the image and

Parameters:

image - Next image in the sequence

updateTrackLocation

protected void updateTrackLocation(T image)

Find the target inside the current image by searching around its last known location

subpixelPeak

protected void subpixelPeak(int peakX,
                            int peakY)

Refine the local-peak using a search algorithm for sub-pixel accuracy.

performLearning

public void performLearning(T image)

Update the alphas and the track's appearance

dense_gauss_kernel

public void dense_gauss_kernel(double sigma,
                               boofcv.struct.image.GrayF64 x,
                               boofcv.struct.image.GrayF64 y,
                               boofcv.struct.image.GrayF64 k)

Gaussian Kernel with dense sampling. Evaluates a gaussian kernel with bandwidth SIGMA for all displacements between input images X and Y, which must both be MxN. They must also be periodic (ie., pre-processed with a cosine window). The result is an MxN map of responses.

Parameters:

sigma - Gaussian kernel bandwidth

x - Input image

y - Input image

k - Output containing Gaussian kernel for each element in target region

circshift

public static void circshift(boofcv.struct.image.GrayF64 a,
                             boofcv.struct.image.GrayF64 b)

imageDotProduct

public static double imageDotProduct(boofcv.struct.image.GrayF64 a)

Computes the dot product of the image with itself

elementMultConjB

public static void elementMultConjB(boofcv.struct.image.InterleavedF64 a,
                                    boofcv.struct.image.InterleavedF64 b,
                                    boofcv.struct.image.InterleavedF64 output)

Element-wise multiplication of 'a' and the complex conjugate of 'b'

computeAlphas

protected static void computeAlphas(boofcv.struct.image.InterleavedF64 yf,
                                    boofcv.struct.image.InterleavedF64 kf,
                                    double lambda,
                                    boofcv.struct.image.InterleavedF64 alphaf)

new_alphaf = yf ./ (fft2(k) + lambda); %(Eq. 7)

gaussianKernel

protected static void gaussianKernel(double xx,
                                     double yy,
                                     boofcv.struct.image.GrayF64 xy,
                                     double sigma,
                                     boofcv.struct.image.GrayF64 output)

Computes the output of the Gaussian kernel for each element in the target region k = exp(-1 / sigma^2 * max(0, (xx + yy - 2 * xy) / numel(x)));

Parameters:

xx - ||x||^2

yy - ||y||^2

get_subwindow

protected void get_subwindow(T image,
                             boofcv.struct.image.GrayF64 output)

Copies the target into the output image and applies the cosine window to it.

getTargetLocation

public georegression.struct.shapes.RectangleLength2D_F32 getTargetLocation()

The location of the target in the image

getTargetTemplate

public boofcv.struct.image.GrayF64 getTargetTemplate()

Visual appearance of the target

getResponse

public boofcv.struct.image.GrayF64 getResponse()