Location New York, Madrid, Barcelona
Its flexibility doesn’t end there. The New York-, Barcelona- and Madrid-based Wildbytes recently leveraged the power of ftrack to tackle an altogether different kind of project; one that blended technology with artistry through a vividly beautiful live facial projection-mapping performance.
When it was approached to work on the launch show of a new line of beauty from Sephora and tattoo artist Kat Von D, Wildbytes sought to create something totally new and totally amazing. And that’s exactly what they did, executing the first-ever live facial projection mapping performance. Throughout the launch show, stunning images were displayed across Kat Von D’s face as she unveiled the line on a stage in Spain. Not only did it wow everyone in attendance, but the resulting campaign reached more than 90 million people as the video footage spread rapidly across the internet.
ftrack was a crucial ally in bringing the performance to life. Despite it being the very first time Wildbytes had used the project management platform, the team came away suitably impressed with its abilities.
Co-founder and Managing Partner, Wildbytes
“Having used ftrack for such challenging project and achieved such success, we would definitely recommend it.”
A wild idea
Julio Obelleiro, co-founder and managing partner, describes the Wildbytes team as a “blend of technologists, artists, and strategists eager to push the field of experiential design through physical-digital technology.”
And that constantly push forward means Wildbytes often finds itself tackling new and surprising challenges, with little opportunity to rest on its laurels.
“We pride ourselves with creating experiences the world has not seen before, and we’re serious about it, which means that we work on many ‘first-ever’ type projects that involve a huge amount of research and prototyping,” asserts Obelleiro. “In the end it’s definitely worth it, for the brands and for ourselves. It means we often work on projects that we have never done before, and that makes them more interesting.”
Sephora approached Wildbytes about the Kat Von D Beauty line, asking the agency to pitch one of its wild ideas to drive huge attention to the debut. Wildbytes, in turn, suggested the concept of a live facial projection mapping performance to anchor the campaign, using Von D’s own face to display memorable and truly beautiful ideas. Sephora was immediately sold on the premise and the unique experience it would present to all those in attendance.
“Sephora was in love with the idea, and even more importantly, they were bold enough to take the risk and jump into such an ambitious project,” remembers Obelleiro. “We were as excited about it as we were scared. There was an intense challenge ahead.”
Wildbytes began with two 3D scans of Von D, capturing her likeness using structured light and photogrammetry to develop an incredibly realistic 3D model, complete with skin effects.
From there, Wildbytes 3D printed a bust of her face, using it during testing and development of the projection mapping effects, albeit without the added challenge of a live, moving face. Following that, the team needed to launch into the colourful, kaleidoscopic imagery that would be projected.
“On the content production side of things, it was pretty much like creating a visual storytelling animation, just on a different output media, from storyboarding to 3D and 2D production, animation, and finishing,” says Obelleiro.
Obelleiro’s team worked on the campaign for four months. An entire new pipeline had to be created to power the performance, including the facial tracking, with five cameras following Von D’s every movement at 240 frames per second.
“We developed a framework that included the real-time face tracking system and a 3D real-time rendering engine, which projected the created content onto the face and adapted to its movements in real-time,” says Obelleiro. “Tons of computer vision and rendering algorithms were needed to achieve the final results and get the system to work well.”
Challenges included the need to consider the distortion of the projector lens (“There is a lot of math involved,” Obelleiro admits), fighting latency issues in the real-time projection, and figuring out which kinds of 3D images projected best upon Von D’s face while onstage.
ftrack’s visual edge
Not only was the project an exciting new area for the Wildbytes team, but the project had a great many moving parts. A unique solution was needed to help keep the project in check.
“This was the first project for which we had used ftrack, and we’re really happy that we did,” explains Obelleiro. “ftrack’s focus on the visual side of things was key for us. Its toolset also helped significantly in streamlining the production timeline and the review process. And this was a very fast-paced project, so the impact was huge.”
It was ftrack’s advanced version browsing features that ended up being the team’s favorite feature.
“This project was about experimentation: projection over a person’s face leads to interesting and unexpected results – sometimes surprisingly great results and others awfully awkward,” admits Obelleiro. “To compensate for this complexity, the versions section of ftrack allowed us to see all the tests at once, meaning we could filter faster in the production process.”
ftrack proved to be a vital tool across the Sephora project. “Even before we launched into actual production, ftrack was having a huge impact in the early R&D phase,” he explains. “It connected the whole team faster in terms of sharing visual results when we were creating so many new visual experiments per day. It could have been easy to lose track of what was going on without ftrack on our side.
“Visual project management tools are a must for production studios, and honestly, the decision is not easy if you haven’t used any software of this sort before,” he adds. “But after having used ftrack for such challenging project and achieved such success, we would definitely recommend it.”
Obelleiro says the team fully intends to use ftrack again in future: “We definitely plan on using it on projects of this nature, where our project management needs go beyond ‘spreadsheet-like’ tools and require something more visual,” he notes.
And there could be many more of them on the way: Wildbytes is further refining the facial projection mapping tech to be even more dynamic, and they hope to push into the music industry, using it for more live performances and music videos.
Wildbytes is also working on further augmented reality experiences, smart mirror installations for retail stores, and a variety of other out-of-the-box and immersive ideas that stick with people long after the moment of contact.
“Brands and their fans are more and more eager to live and see new experiences, and we will continue crafting ideas and innovation-based projects and campaigns for those clients willing to experiment,” Obelleiro affirms. “2016 couldn’t be more exciting for Wildbytes, and with ftrack on our side, we can’t go wrong.”