Personalised welcome message via facial recognition

At Duke & Grace we organise a Trend Event for our customers every year. But this time, we wanted to do something special. It was our goal to show the visitors how well facial recognition works, and how easily it can be implemented with a limited set of data. The idea was to give the attendees a personalised welcome message when entering the office.

Face recognition

We decided to work with this Python library. They built a deep learning model that recognizes face landmarks on a photo. Next, it calculates the distance between those landmarks. The resulting figures are then compared with the landmarks on the attendee’s profile picture we found on the internet.

You’d be surprised how easily you can find a usable reference photo to practice face recognition. Nowadays, basically everyone has a suitable profile picture. Even if that photo is quite small, the only thing we needed were the face landmarks.

Our Python script processed the video feed from an IP camera frame by frame. If someone was recognised, a request was sent to our CMS (Harmony).

Harmony

Harmony is our tailor-made CMS based on Symfony. Since there are some core components already available, we could set up a CMS very fast. The account managers could easily add attendees and write a personalised welcome message.

We also made it possible to manage the templates used on the TV screens and dynamically assign available slots for personal messages.

Socket.IO

We’ve used the browser of the TV to run our web application. Via WebSockets and a socket.io server we were able to display the personal message in realtime. Every TV screen was connected and had available ’slots’. Our application was able to fill these slots with a message. After 10 seconds the message disappeared and the slot was available for a new message.

If a person was recognised we sent a request to our CMS and the CMS fetched the personal message from the database and sent it to a free slot on the TV. We’ve put some extra work into it to make this process as fast as possible. From recognizing the visitor’s face to seeing the response on TV, we aimed for 2 seconds.

Design Studio, a special approach

You might have a very specific challenge you want to be solved. Our dedicated Design Studio focuses on just that.