OpenCV and Processing 14 – Magic & Love Interactive

This example continues to explore the Video module in OpenCV. It uses one of the BackgroundSubtractors, the BackgroundSubtractorKNN. It learns the motion in front of the camera and treats the stationary scene as background.

In the code, the important command is

bkg.apply(frame, fgmask);

The subtractor object bkg takes in the latest frame and generates a foreground mask, fgmask. We can use the foreground mask to single out the foreground moving object.

Here are a number of screen shots from the testing. The background is the planet Earth image from NASA.

import processing.video.*;
import org.opencv.video.Video;
import org.opencv.core.CvType;
import org.opencv.core.Scalar;
import org.opencv.core.Mat;
import org.opencv.video.BackgroundSubtractorKNN;
 
Capture cap;
CVImage img;
PImage earth;
BackgroundSubtractorKNN bkg;
Mat fgimg, fgmask;
 
void setup() {
  size(640, 480, P3D);
  background(0);
  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
  cap = new Capture(this, width, height);
  cap.start();
  cap.read();
  img = new CVImage(cap.width, cap.height);
  bkg = Video.createBackgroundSubtractorKNN();
  fgimg = new Mat(cap.height, cap.width, CvType.CV_8UC4);
  fgmask = new Mat(fgimg.size(), CvType.CV_8UC1);
  earth = loadImage("background.jpg");
}
 
void draw() {
  img.copy(cap, 0, 0, cap.width, cap.height, 0, 0, img.width, img.height);
  img.toCV();
  Mat frame = img.getBGRA();
  bkg.apply(frame, fgmask);
  fgimg.setTo(Scalar.all(0));
  frame.copyTo(fgimg, fgmask);
  img.fromCV(fgimg);
 
  background(0);
  image(earth, 0, 0);
  image(img, 0, 0);
  text("Frame rate: " + nf(round(frameRate), 2), 10, 20, 0);
  frame.release();
}
 
void captureEvent(Capture _c) {
  _c.read();
}