We decided to work with this Python library. They built a deep learning model that recognizes face landmarks on a photo. Next, it calculates the distance between those landmarks. The resulting figures are then compared with the landmarks on the attendee’s profile picture we found on the internet.
You’d be surprised how easily you can find a usable reference photo to practice face recognition. Nowadays, basically everyone has a suitable profile picture. Even if that photo is quite small, the only thing we needed were the face landmarks.
Our Python script processed the video feed from an IP camera frame by frame. If someone was recognised, a request was sent to our CMS (Harmony).
Harmony is our tailor-made CMS based on Symfony. Since there are some core components already available, we could set up a CMS very fast. The account managers could easily add attendees and write a personalised welcome message.
We also made it possible to manage the templates used on the TV screens and dynamically assign available slots for personal messages.
We’ve used the browser of the TV to run our web application. Via WebSockets and a socket.io server we were able to display the personal message in realtime. Every TV screen was connected and had available ’slots’. Our application was able to fill these slots with a message. After 10 seconds the message disappeared and the slot was available for a new message.
If a person was recognised we sent a request to our CMS and the CMS fetched the personal message from the database and sent it to a free slot on the TV. We’ve put some extra work into it to make this process as fast as possible. From recognizing the visitor’s face to seeing the response on TV, we aimed for 2 seconds.