Facial Drone Stabilization using a Single Onboard Camera


Team: Mezan .M. Roy, Zohar Roy

Today’s toy quadcopters are great. They are quite popular, due to the good balance between low cost and capabilities. However if one searches for a drone with that can accomplish basic tasks, prices skyrocket.

The goal for this project was to modify a simple drone, and get great tracking results for a fraction of the price in the market.

Our project was to create a drone that follows a person. We based ourselves on a slightly similar project that has already been done:

Drone Stabilization using a single onboard camera



Main loop of the project

  1. Images from the drone are sent to the computer

  2. The computer searches for a face
  3. The computer finds how the drone needs to move relative to the face
  4. The computer sends instructions to the drone

The program is written in C++. We used the OpenCV library.

For information about how to compile the code, find the README.md file found in the source code.

Face recognition

We needed a fast and efficient face recognition for this project so that we can still maintain a high frame rate for stabilization.

We used OpenCV’s Haar Cascades, which yield fair results pretty quickly.

Stabilizing our drone requires getting instructions at 20 frames per second at least. Unfortunately, the fastest face recognition we found can run no faster than 10-15 frames per second. We solved this problem by creating 2 threads:

  • A main thread that sends instructions to the drone. As face recognition, it uses a simple correlation from an updated template from the other thread.
  • A “side” thread, that does the real face recognition and updates the template that the main thread uses.

Using this technique, we were able to increase our frame rate to around 20-25 frames per second, which is enough for drone stabilizing.

3D Location Calculation

Based on the position of the face in the picture, we understand the drone’s position relative to the face and understand where it needs to move.

We used another OpenCV method called solvePNP.
This method takes a set of 2D points, and a matching set of 3D points in the world, and outputs the 3D transformation needed to make the camera see the points as 2D points on the screen. The 2D coordinates found using image processing, are given as input to this method.


Controlling the drone is done by sending instructions using the following directions:

Using the transformation found from the solvePNP algorithm described, we use the PID method to translate the 3D transformation into these 4 controls.

Close Close