ResonanceDesign

Image face detection and facebook thumbnails

In the process of a project that required me to think of ways to do something interesting with facebook profile pictures, I had the challenge to predict, where facebook is cropping it’s thumbnails. It easily becomes very clear that they are using some sort of face detection and a few extra rules to make sure the faces in a profile image appear in the best way possible on the thumbnails. I thought I publish my findings to make other people’s life easier.
Although facebook’s cropping mechanism is fairly good, it’s not perfect or not necessarily logic, however here is how I reversed engineered it:

FACE DETECTION
I used face.com’s API to get the initial faces. Go there, get yourself a free API key and you will also see that face.com does way more than just simply detecting a face, but that’s another story.

REALISTIC FACE SIZE
In order to get the actual face, the basic rectangle needs to be enlarged. That is the original face detect recrangle is only cover- ing the eyes all the way down to the mouth and a bit of chin sometimes. But we want forehead, hair, etc. as well. Therefore I, and seemingly facebook as well recalculates this average rectangle on all sides, mainly the area above the eyes.

MAKE BEST USE OF ORIGINAL IMAGE
Now you probably have noticed that on some images, people’s faces are fairly small even in the thumbnail, meaning the thumbnail doesn’t try to zoom in too much. The main reason for that being that if you would do so, the thumbnail would pixelate badly. So facebook is using 85% of the image, given the lower side width or height to produce a square. You will find that even images that are the exact square of the face itself, are cropped on the edges.
After the maximum square is calculated, it is pushed into the image in case the 85% size goes over the edges of the image itself.

ADDING PROPORTIONAL GRAPHICS
And then the real fun begins. In the example below, I took all that information to place devil’s horns (why?!?) on every face. The face detect’s eye locations and yaw, roll and pitch help identify angled faces, and the gender property also from the face detection help me place different horns on men and women. For good measure, I put Simon Pegg all resized and smily next to the person most in the front, just the way he stood next to me at that Iron Man premiere…
The large red rectangle shows the predicted crop of the image, this is where you want to place modifications, if they should show in the thumbnail. It seems facebook crops the very center, if there are no faces and finds the middle of all centers, if there are multiple faces.
Given all that the code can make a much better experience than what Twibbon does and I have only started. The code is not per- fect, it will not work very well with sideways faces and covered faces, but that’s something I could fix, given I had more time on this. But as a proof of concept, I hope this one helps. It’s also not the most creative use of the tech, but imagine what other cool graphics and situations one could build with this.

DEMO
I have a demo uploaded at http://www.resonancedesign.co.uk/cgi/php/profileModDemo/ where you can see it in action. If you want to test it on yoru facebook profile picture, go to your facebook account, right-click the image and copy the image location. Then add that URL to my demo URL like: http://www.resonancedesign.co.uk/cgi/php/profileModDemo/?sourcePath=mypath.jpg (e.g. http://www.resonancedesign.co.uk/cgi/php/profileModDemo/?sourcePath=http://profile.ak.fbcdn.net/hprofile-ak- snc4/186936_655357406_4468690_n.jpg)

NJoy!

Be Sociable, Share!

•