Innovation at its Best: The Story Behind Enter Black’s Project

Project

Enter Black had produced a project called the “Metamorpheur”. This was a Photo Booth in which a video projector projected filters (like Instagram filters) directly onto people’s faces. Flynx was to bring this project to V2 by adding face tracking so that the projection would follow people’s movements.

Technical issues

Face tracking

Facial tracking is carried out using an algorithm based on machine learning. It allows us to obtain 478 facial landmarks. The difficulty in tracking here lay in two points: The darkness caused by the photo booth. The image recovered by the camera is very dark. Detection when a filter is projected onto the face. Projection of the filter greatly blurs the camera image and analysis is more complicated. To limit these risks, the first two weeks of development were devoted to benchmarking different algorithms in these complex environmental conditions.

Camera and projector calibration

To transform the pixel position of the face as seen by the camera into a position in the image projected by the video projector, the camera and video projector had to be calibrated to determine the translation and rotation between them.

We developed tools to perform these various calibrations. This was the crucial and most complex part of the project. Without these algorithms, the projection would never have been properly aligned with the face.