The last post demonstrated the use of the face detection function in MediaPipe with TouchDesigner. Nevertheless, it only produced an image with the detected results. It is not very useful if we want to manipulate the graphics according to the detected faces. In this example, we switch to the use of Script CHOP to output the detected face data in numeric form.
As mentioned in the last post, the MediaPipe face detection expects a vertically flipped image as compared with the TouchDesigner texture, this example will flip the image with a TouchDesigner TOP to make the Python code simpler. Instead of showing all the detected faces, the code just pick the largest face and output its bounding box and the position of the left and right eyes.
Since we are working on a Script CHOP, it is not possible to connect directly the flipped TOP to it. In this case, we use the onSetupParameters function to define the Face TOP input in the Custom tab.
def onSetupParameters(scriptOp):
page = scriptOp.appendCustomPage('Custom')
topPar = page.appendTOP('Face', label='Image with face')
return
And in the onCook function, we use the following statement to retrieve the image from the TOP that we dragged into the Face parameter.
topRef = scriptOp.par.Face.eval()
After we found out the largest face from the image, we append a number channels to the Script CHOP such that the TouchDesigner project can use them for custom visualisation. The new channels are,
- face (number of faces detected)
- width, height (size of the bounding box)
- tx, ty (centre of the bounding box)
- left_eye_x, left_eye_y (position of the left eye)
- right_eye_x, right_eye_y (position of the right eye)
The complete project file can be downloaded from this GitHub repository.