Optimizing Player Tracking: Innovative Solutions for Smile in the Light

Tracking algorithms

Projector calibration

Cameras and projectors are never perfectly aligned. So you need to be able to calibrate the cameras with the projector, so that when a camera sees a person or a point, you know ‘where’ on the projector screen that person is. This is crucial in order to make sense of the position tracking. Fortunately, we have developed an algorithm that calculates this calibration in a fraction of a second. All the cameras have to do is see the projector image, and in just a few frames they know exactly where they are in relation to the projection screen. It’s a bit like our magic formula, so we won’t dwell on the method any further…

Point cloud analysis

As we’ve seen from our work on projects using depth cameras (particularly the Kinect), the ‘body tracking’ part is often extremely costly in terms of GPU and creates latency. What’s more, we don’t always need to know the position of every member of the player’s body, but often only their position on the ground. In general, we therefore analyse the point cloud directly on camera output. The first step is often to capture a ‘baseline’ of the room, which we use to detect any differences between what is being scanned and this baseline. These differences are then fed into a ‘blob’ detection algorithm. We look for all the clusters of points that are close together. Finally, the last stage is the tracking itself. We keep a history of the ‘blobs’ in order to associate new positions with old ‘blobs’, to detect if a new player arrives or if a player leaves. For this project, the entire algorithm is written in C#, so that the client can get a Unity package that can be directly integrated into the game.