Episode 5 (Nov 2017) – Augmented Reality & Holograms in Entertainment – Part 1

In today's episode our CEO Alex, Chief Product Officer Xava and CTO Norbi talk about the technologies that allow any existing or imaginary character be brought to life in real time. They will dispute the existence of holograms and discuss the role of character licensing.

Time-stamped show notes:

[00:42] Q: Emerging technologies now allow real-time “resurrection” of any character, anywhere. What are some of the most interesting examples of bringing characters to life and what are the technologies behind them?

[00:56] [Norbi] The most interesting one for me – although it’s not real-time – is General Tarkin in Star Wars, Rogue One. The actor who played this character passed away a long time ago, so they digitally resurrected him for “Rogue One” […] The technology couldn’t really get all the way to the other side of the uncanny valley – the character looked good, but you could tell that he’s not real [...] In the Planet of the Apes movies, you don’t think for one second that Caesar is not real, but that’s because he’s not human [...]

[2:28] [Norbi] The technology behind the first example was that they collected all kinds of different footage of the actor, digitally recreated his face, found an actor who looked pretty much the same, and using facial motion capture they mapped all his facial movements onto the digitally recreated face [...] The technology is not there yet for human characters.

[3:27] [Xava] Tupac [...] There was this concert at Coachella [...] Everyone knew about the “Tupac hologram” – it was a simple projection method, and wasn’t real-time, so they created an animation of Tupac singing, you could see that it wasn’t as fluid, but what was interesting that he was, interacting with Snoop Dogg on stage. The result was like a glimpse of what the future can be [...]

[04:55] [Norbi] What’s interesting for us in the AR space is that (again) in Star Wars there was an android robot – in the final movie they used footage of it rendered out straight from Unreal Engine, a real-time game engine, which is very promising.

[05:51] [Alex] To go back to the Tupac one – that was projection foil [...] They attempted the same thing with Michael Jackson [...] The problem is that no one is going to buy into it as much if it’s not the footage of the person. So if it’s a 3D slightly uncanny, weird representation of that person [...] That’s really difficult, because the sell is that he has come back to life – but if he hasn’t come back to life, there’s not much of a sell.

[08:48] Q: INDE has received numerous requests to create 3D holograms – do they even exist?

[09:02] [Alex] They don’t exist. I don’t see any downturn in the amount of requests, but one thing I’ve noticed in the past year or two has been an added layer to that. There are people who are willing to forget that they know that holograms don’t exist, and carry on the conversation anyway [...]

[09:55] [Xava] Just to be clear on what a hologram is, let’s go back to the Star Wars reference [...] A hologram needs to appear in a 3D space without using any devices [...] Not something that is commercially available.

[11:15] [Alex] There are things out there that create the illusion of that happening, AR is one of them. [...] It creates the appearance of the object that isn’t in the real space, that’s why as an Augmented Reality company we tend to get requests for that kind of thing [...] The other thing that we’ve seen is holographic foil, which is a technique that is approximately 150 years old [...] And then there’s things like projection onto steam, which has been interesting to watch. On the 30-second portfolio movie it looks incredible – on the ground you’ve just got a hugely elaborate setup with huge amount of limitations [...] Every time you get to the point when someone says “I want a holographic representation of X”, there’s a complete misunderstanding about how much infrastructure needs to go in to even create something that gives the illusion of that.

[12:57] [Xava] I have a great example of how the audience are willing to just believe in the illusion, even if the setup is not complicated – it’s popstar Hatsune Miku [Facebook page], a virtual singer in Japan. It started as a plug-in for audio production, a voice synthesizer. They started to brand it with anime figures, so one way to sell it was to create a song with this virtual pop star singing, using that plug-in [...] Hatsune Miku is now doing concerts, she has movies, records – a full industry about the virtual character that goes on tour.

[14:10] [Alex] That’s an interesting approach, as that character hasn’t existed in the real world anyway, so you are willing to meet and interact with that character in different ways.

[14:29] [Norbi] Is it similar to Gorillaz [Facebook page]?

[14:32] [Xava] Exactly, it was before Gorillaz. We know that the Japanese culture is more willing to accept these kinds of things, and then you don’t mess with the “uncanny valley” approach – it’s a virtual character by definition. It’s interesting because the physical boundaries are not there, you can play more with these things [...]

[15:07] [Alex] The Gorillaz example – actually, they’ve remained 2D characters for a tremendous amount of time [...] To transfer characters from 2D to 3D is incredibly difficult [...]

[16:12] [Xava] Just to see the full history of Gorillaz in this aspect – it’s just amazing. I just saw their concert two weeks ago in Vienna. You don’t know what to expect, you have all these image sin your head. Why it works is because the band is an idea. The first public appearance they had was at 2005 MTV Europe Music Awards, and again at the 2006 Grammy Awards, with the addition of Madonna. They used the same principle as Hatsune Miku and Tupac, the same foil – it was the first time that they appeared in 3D and that’s been the ever-lasting impact [...]

[17:50] [Alex] With all of the holographic stuff in terms of projection foil – having had conversations with people in various places around the world, who actually have them, have the screens and they try to build performances around them, they ran into some really interesting issues. One was about having physical characters running around the stage that was supporting the holographic foil, so if anyone moved, the floor flexed and made the digital character flex slightly, breaking the illusion.

[19:12] Q: How does licensing play a part in the idea of digital resurrection?

[19:25] [Alex] One element that is still massively missing in the AR industry is something the VR industry has been quicker to pick up – the principles of TV, distribution and licensing had been applied to VR almost immediately [...] The content being good comes down to me to two things: 1.) the understanding of how to make content for that medium and why it’s useful and 2.) having content and potentially IP in there that people identify with – and I think that’s a much harder play, especially with the AR industry because it’s just incredibly early on [...] There’s not an immediate translation from VR into AR in terms of how we’re going to use it.

[21:12] Thank you for listening. If you’d like to know more about INDE, please visit us at www.indestry.com


Music Winter Boulevard by Minuit De Lacroix
Photo Mike Wilson