Episode 6 (Dec 2017) – Augmented Reality & Holograms in Entertainment – Part 2

In our previous episode Alex [CEO], Xava [CPO] and Norbi [CTO] talked about the real-time digital “resurrection” of characters. Today they discuss other interesting aspects of AR-based entertainment such as the importance of content versus technology, and the limitations and misconceptions. They will also take us through some creative real-life examples. We hope you enjoy.



[00:46] Q: Norbi, is content as important as technology when it comes to the final result in AR entertainment?

[00:52] [Norbi] It’s more important, I would say. The technology is really important – AR gives you the perfect illusion (tracking, recognition etc.). Once you see a virtual content floating in real life it breaks the illusion, but if the content/character is fixed to the real environment then you will have the illusion of a hologram or a real object.

[01:29] [Alex] You just said the word “hologram” again… [laughter]

[01:31] [Norbi] I’m sorry about that, it won’t happen again. [laughter]

[01:40] [Alex] You’re part of the problem, not part of the solution. [laughter]

[01:45] [Norbi] So about content – content is king in AR. If you see a beautifully created object or character in a real-life environment, lit in the same way as by real light, you will instantly think it’s real. Also, if you have a perfect animation with a not-so-beautiful character (e.g. comic character), but it interact with the real world, again, it’s the perfect illusion… so technology is important to have the perfect mapping and the perfect rendering and the optimal performance in terms of playing out the content, but at the end of the day the user/viewer will only notice the content [...]

[02:51] [Alex] There’s two strands to it – the first strand, i.e. the first phase of the industry that we’ve been is has been “how real can we make the content look”, so if you go back to when we were playing around with mobile in 2009/2010, the holy grail was to have more than 500 polygons [laughter] [...] So the first strand – and the industry still seems slightly too focused on it – is the realism of the content. If the content looks real, if it is shaded in a great way, lit in the correct way, like you said, with ambient light, then you can start to give the illusion of the content being there [...] But then the bit that’s still missing is the storytelling, the narrative bit… so you’ve got 10 years worth of focusing on rendering which is only really providing a platform for you to deliver a model that looks like it’s here [...] Like our endless conversation about live-rendered water, but then again, even if you can make water look real, what story are you trying to commmunicate? [...]

[05:01] [Xava] The medium is in an early stage where you are like “Okay, what are the patterns on AR narrative?” [...] One part is how you relate the digital content to the physical environment, how you play with all the elements in order to create a physical scene, a physical narrative, and that’s when it starts to make sense [...]

[05:55] [Alex] Also, there’s not really any documentation, there’s not a lot of testing... We’ve built a lot of location-specific stuff in AR and we’ve made mistakes..  and we’ve pushed things forward sometimes, as well, but I feel like no one’s able to categorically say “this is how people will interact with this piece of content / this performance in a public space [...] You can’t google it, you can’t go and find a hundred examples of how people interact with a character that isn’t there, using a language they don’t understand, in a place they have never been to before – its probably the most difficult thing, actually.

[06:53] [Xava] I totally agree [...] When we are receiving requests, the reference that our clients have is “I want something that people will always come back to [...], so they want to have retention in an AR environment, which is difficult to achieve [...]

[08:04] [Alex] To assume that the Pokemon Go example was a success because of AR is to completely misunderstand all of the game design that went into it. That franchise alone has enough power to carry something anyway. The AR module was basic, it was new to people who’d never experienced it before, but the game design was what made it. The game design and the IP. [...]

[09:19] Q: You guys mentioned Gorillaz. With musical acts like them, Run the Jewels and others hopping on the augmented reality bandwagon, do you think AR could kill the video star?

[09:34] [Xava] Those are really good examples about how you can integrate AR in a way to make your audience discover you, discover your content, your music, your brand [...] Gorillaz, when they launched their latest album, they released an application that was more like an experience, with two ways of viewing – an AR view and a VR view. I think that it was brilliant in a way that it set the mood of what kind of album you’ll expect. It was part of a marketing campaign, it created some buzz around them.. Was it super useful? No. But it didn’t need to be. They understood really well what was the advantage of delivering the message in that way. With Run the Jewels it’s more like and old school AR thing – with the album that they are releasing they have a companion app, where you can see exclusive content only if you have the physical object. So it opens a lot of options on the marketing side if you understand the medium, and what are the extras that you are delivering to your audience. [...]

[Listen to the podcast for some really cool details on the Gorillaz example]

[16:44] [Xava] Nothing can beat a live performance of a musician, but the rest of the communication is made through digital content, which, is brilliant because you can make it work in different spaces. [...]

[17:31] [Alex] One idea that floated around – and I would imagine that Norbi and I will probably end up having conversations about this – was the idea of using the devices at a concert to trigger extra layers of content, so augmenting in a very normal way, augmenting the experience you’re already having. That I haven’t seen a lot of. We specced it a couple of times, but it was very early on… early times in terms of the technology and what it could do. The idea of you being able to look up and see a human performance and then have layers of stuff added – there were conversations around the animation reacting to the speed of the music –, that gets really interesting. [...]

[18:40] Q: What are the technological limitations and common misconceptions?

[18:48] [Norbi] Misconceptions... we just touched on that – “it’s a hologram” [laughter]

[18:55] [Xava] There’s one thing – if we’re talking about AR in entertainment [...] All the examples we’ve discussed (Tupac hologram, Gorillaz etc.) were linked to a physical location, to a stadium, to some place they were playing, so you’re replicating something in AR that won’t be better than the real thing. Where is the opportunity with AR? When you detach that physical space and you spread – so instead of ten thousand people going to see Gorillaz, you can put Gorillaz in different parts of the world. [...]

[20:06] [Alex] I think the limitations – when it comes to something like mobile – are still based around amounts of content, types of content, how it looks, how much experience you can have and for how long… There’s just massive misconceptions in terms of how the application actually works, I think there’s even misconceptions in the industry, misunderstandings in the industry sometimes, as well. [...] It’s almost a continuous conversation about all of the things that you can’t do [...].

[Listen to the podcast to find out more about misconceptions and limitations]

[25:13] [Alex] [...] And then to me the other strand is the ideas that appreciate that they can only really come to life properly when they’re on a full-blown wearable device. I’ve probably referred to it already in this series – someone walking up to something that we built on mobile 4-5 years ago and saying “this will be great when I don’t have to hold an iPad”. So perfectly described. [...]


Music Winter Boulevard by Minuit De Lacroix
Photo Mike Wilson