Depth Piano

3D cameras project

Yarin Didi, Inbar Pinsly, Sean Man

The Problem: detecting key pressing and the amount of pressure the player apply to each key
Why not 2D? to capture accurately the pressure the user apply to each key in 2D, we need to place the camera in the place of the player body – not feasible.
Our Approach: the depth-camera is placed above the keys (in this way the camera doesn’t interfere the player). at the beginning we set for each key a virtual plane (Using PCA for estimating the normal vector to the plane from the 3D points) and for every frame we estimate the current plane that defined by the keys and check the angle defined between the ground plane (from the beginning) to the current plane, if we passed certain threshold – it is a key pressing. notice that we can tell that amount of pressure applied to the keys by taking the derivatives of the angle (along the frames).
Implementation: our algorithm was implemented using C++ supported by the OpenCV library and the RealSense SDK for Intel Depth Cameras (a.k.a RSSDK).

algorithm

 

Piano

Accessibility
Close Close