https://www.linkedin.com/in/mbuckwald

This interview with Michael Buckwald is all about the future of hand tracking. Michael is the co-founder and CEO of Leap Motion. The Leap Motion Controller is an innovative hand tracking device that allows users to interact with digital content using their hands in virtual reality, AR, and other computing platforms. Developers can use it to create applications around hand movements. When you try it out for the first time it feels a little like magic. Leap Motion has also released a version of their hand tracking software especially for VR, called Orion. That could be a game changer.

I invited Michael to the show to learn more about his background and what’s going-on at Leap Motion.

Here are other things we talked about:

-How did you start Leap Motion? Where did the tech come from?
-What do you want it to do 5 years from now? What tech hurdles do you have to overcome?
-Tell us about Orion? Why is it significant?

Transcript

David Kruse: Today we are lucky enough to have Michael Buckwald with us. Michael is the Founder and CEO of Leap Motion. Leap Motion is an innovative hand tracking device that allows you to develop applications around hand movements. When you try it for the first time, it feels a little like magic or maybe a lot like magic, all right, but now Leap Motion is come up with a hand tracking project for virtual reality called Orion, and which could be a game changer. So I’m excited to learn more about that as well. So I invite Michael to this show to learn more about his background and what’s going on at Leap Motion. So Michael, thanks for joining us today.

Michael Buckwald: Yes, thanks for having me here.

David Kruse: And so let’s – before we dive into Leap Motion, could you tell our audience a little bit about your background, it’s interesting.

Michael Buckwald: Yes, so you know I think – well, I founded Leap Motion with David Holz. He’s our Co-Founder and CTO and he and I had grown up here in South Florida and David was getting his Applied Math PhD and I was in Washington DC when I started another startup called Zazuba which was a sort of online business space startup. And after I had sold that and Dave sort of felt like his technology was good enough that he wanted to drop out of his Math PhD and start doing real things we joined forces and move out to San Francisco and that was really at the beginning of a long journey definitely. What is now I guess a six or seven year journey with a lot of us known, but it’s been very, very rough.

David Kruse: And when you started being friends in South Florida?

Michael Buckwald: Yes, so Dave and I were friends probably as early as like late elementary school, through high school.

David Kruse: Wow, that’s a good story, that’s a good story.

Michael Buckwald: So there’s been a lot of history. We’ve known each other for a very long time.

David Kruse: Yes, I guess you guys know each other and you’re still together, so that’s worked out well and still friends.

Michael Buckwald: Yes, we are still together, yes.

David Kruse: Yes, at least in the business sense, let’s say.

Michael Buckwald: Yes and you know obviously it’s very – I think it’s very important when you start a company, especially something as hard as Leap, that I think any start up is difficult enough, you really have to have your people around you that you can trust, because there just so much. You can’t even just – its pulling off of many different things with that regard and parallel starting off its and certainly impossible if you got to sort of trust the people around you. So, that’s definitely a benefit to starting a company with someone that you’ve known for a long time.

David Kruse: Definitely, and you need somebody at the startup where you can – you might trust them for a little while and then realize that you don’t trust them. But you’ve known – you guys have known each other long enough to know that you trust each other.

Michael Buckwald: That’s true, yes it’s definitely been enough time that there is that trust there.

David Kruse: Nice and before you talk about Leap Motion, I was curious, you were involved with the One Laptop per Child project, which is interesting. How did you get involved with that?

Michael Buckwald: You’re talking about the Zazuba website.

David Kruse: Yes. Oh yeah, yeah, yeah or maybe you were not involved with One Laptop were you?

Michael Buckwald: One Laptop. Sorry I presumed – I think I set up a web product.

David Kruse: One laptop per child.

Michael Buckwald: So bold that, so there was One Laptop per Child and it was a great experience and that was right out of college. I moved out to Madagascar and spent about six months there running the first deployment of laptops for OPC [ph] in Madagascar and then stuffs there is Africa. That was a really great experience and I think I would definitely sort of – it was a good reminder of the power of technology and how you know when you give someone a piece of technology suddenly they are providing access only to sort of things they observe about the media world around them to bring out all information that’s been collected direct in history and I think that, that’s really been our driving process to be here; it’s not say been about tracking hands so that people can do, they can pick up blocks or something. It’s been about making people, making humans more connected to technology in the broadest possible service, so that each can be better and it gets back to you know how with those kinds went from almost no access to the history of the human race and all of its generally. Now it is there and now it’s available for everyone, including the kids, not everyone here through Smartphone’s, but I think the next question is sort of why hasn’t it evolved beyond you know input as a sort of binary yes or no, where even with something like a Smartphone where now all the world’s information is in their hands, we are either touching it or not touching it. And that proved that it made not that much different than a keyboard where you are pushing a key a not or even relatively that to a punch card, it’s not really – I’m not sure if we have been punching the card or not punching the card. So that lets us sort of wonder I guess how do we – what is the missing link and for us it’s been clear that it’s input.

David Kruse: Interesting and that makes sense. So let’s talk about Leap a little bit and how did you or when did you start it and how did you come up with the idea, and the initial technology?

Michael Buckwald: Yes, so I think the idea has existed for a very long time. Really if you go back, even hundreds of years there is science fiction and even before that like there were legends that involved things that are trying to like sort of interface and I think that, that’s really good actually because it features how deeply infringed in our minds is the idea that we get our access to technology in a better way or ways and I think that’s why even people who are not technical don’t think about these problems at all, are still kind of amazed when they see this, and then are therefore during every science fiction move or book and I think that again gets its desire even its unconscious to have more. And so for us it’s not one of the idea so much as how you actually execute on this well, because many, many people, you know many of the smartest people in the world for the past 15, 20 years and certainly for the past junior high, over 10 years, while we started the company have been trying to be risks in real life and it’s a very, very, very hard problem, because you have to have to track hands and figures with incredible accuracy and because really, our role has always been to sort of leverage the fact that at the end of the hands and fingers are really capable of incredibly nuance and sophisticated atrium, because when you think about a very basic everyday action like reaching out and picking up a cup and drinking from it, that goes sort of biops every day because it’s such a basic part of our lives. But if you stop and think about it and you think about all the little movements in coordination, it’s quite magical as I – because it really jumps out of you that we do things like that thousands of times of day and we succeed 100% of the time and everyone does it and it kinds of goes back to sort of millions of years of evolution that have given us the ability to do very, very complicated things with our hands and figures, but then are also very nuanced and powerful. So for us that started this journey which really started with the patent company make down six or seven years ago, of how do you track hands and figures well enough that all the figures, all the movement, all the subtlety, all the speed and all of that is captured and people can forget that the hands that they are seeing that we are recreating digitally are not their physical hands. And anyone could walk up to a “computer” and they can interact just like the real world.

David Kruse: Yes, the first time I used it, I remember thinking to myself, well this is how it should be. It just felt natural and right, and so…

Michael Buckwald: And that’s a good point and one of the things we really believe is important for it to feel natural is for it to be physical or direct, because a lot of people think of this space as sort of gestures and we think of gestures as more like sign language, where if I hold up like a T sign with my finger, then something happens on the computer and the window closed or something. That’s interesting, but that really doesn’t solve the problem because it’s not any different and I personally asked about another keyboard window, but it’s the same binary hand happening somewhere else and in this case these are just memorized a bunch of gestures. But we think that instead if you can, if you can tracks the fingers and hands really fast accurately, it will see their hands, especially as VR or AR are a part of them and the hands just function with the system that they would expect like when they touch a leaf that gently rocks as they would expect it to and they rather bock it back as they would expect it to in real life and that kind of system can be soo much more powerful and it lets people do things that they just never, ever do with a [inaudible] or a gesture, but it also doesn’t wait its much, much more powerful because you don’t have to memorize sign language which people should never have to do and would never do I think.

David Kruse: Yes can you give a use case how – well, that you think is a clever way or interesting way that people have used your technology in some type of application.

Michael Buckwald: Yes, so right now we are posting of VR and AR and we are working with various companies in that space to embed the technology in the actual headset so that especially for some of the mobile types of virtual reality, which really are – where most of the units are being solid now, where you even have the luxury of carrying it around. We think that there are obviously bigger issues than just the fact that they are not portable, but there is also just that. That you have your hands everywhere, but you certainly don’t have a log with you. What you want is that you pull out a posture or something when you are playing. But they have done also everything like a wand. It gets in the way of the Holy Grail, which is really premise, because what everyone has been told and what people deserve I think from VR is this idea like when I put on the headset I walk. I feel like I’m being teleported near products and when you don’t have your hand, when you see like an Xbox controller, although you don’t see it actually, you are just holding the Xbox controller, that’s a huge red flag constantly. Every time you move, every time you touch it that you are not on the surface of Mars walking around, but when you see your hands move exactly as the characters hands move and you see them interact with the environment exactly as they would move in the real space in the real world following those runs, it creates this massive sense of emersion to magi and so I think in general we are focused on creating – we think has for VR sort of the best primary input and for AR also the best primary input, but I think that there are obviously – there are lots of facilities in these cases that we are uniquely attached to. For example, we think that your hands will have a really transformative effect on education, because if you think right now about kids playing ball outside and sort of every kid growing up develops a intuitive understanding of physics of that ball because of that and it’s not something that they can be taught if you ask them to play right up the equation or the equation. But a small child absolutely knows the physics intuitively at lease of bouncing a ball, but in the real world where you are using your hands, can we develop that same physical intuition that stays with us our entire life, but has to be around like quantum galaxies and very complex math and things and can, you know, a five-year-old grow up to have that dosed into their minds forever the same way. That they do things like basketball, and I think that that’s – I think sort of theoretic learning in general. Whether it’s that sort of thing, but with experiencing the things that just aren’t possible to experience right now or these things that are possible to experience but are conveniently getting back in the [inaudible] being able to save it and share it and move it around. I think certainly social VR and AR is an interesting space down there. I think that you know being able to see lots of characters around the world, especially in AR because I think that AR I think will actually be the biggest market. I think VR is going to be very well changing, but I think that AR with companies like magically putting hold-ins and now billions of dollars pointed to the space. I think that we are going to have glasses that led us – where it’s something that is about the size of a regular pair of glasses, but project physically indistinguishable thing around the environment that actually look exactly like we are looking at the real world. I think that implications of that are very, very massive. I think that that has to have great hand tracking to exists, because you will see your hands in front of you and you will have to really interact with the things around you in the world, but I think that’s obviously that at the very least it sort of eliminates all other consumer electronics, because they can project a screen. They will either need a TV or phone or anything, but more than that it even kind of changes their concepts of space and time and like nation states, because kids will grow up with friends from like Bangladesh and Australia and they will like walk down the street and play games like kids do now in real life and you will be able to attend like work meetings and conferences with people all over the world in a progression environment that looks like Switzerland or something, so that’s… The implications of that technology are so massive that I – there is example of just a very, very tip of the iceberg and I think we are happy – we are obviously – it’s very exciting to see sort of the culmination of you know billions of dollars, the output problems being solved now and also obviously the input problem. I can see you have to have hands I think for it to be – for it to create that sense of magic, because it would obviously be. I would be very disappointed if had a pair of almost magical glasses and I could – it is projecting a magical like snow field or something, but when I use my own hand I see though the transparent glasses to touch things that just went through them or didn’t interact at all. That would be the most disappointing thing. So fortunately we’re there to make sure that doesn’t happen and to make sure that those gestures are very hand tracking to you. But it’s a very interesting time and I think we are – this is really just the beginning even for VR, but also there has been that full error inflection point that I think is also entirely sort of order of magnitude and more transformative to the world.

David Kruse: Yes, I think that’s interesting. I can’t wait for that vision to become a – it’s kind of a reality, but not a true reality yet, but I mean you guys.

Michael Buckwald: Yes, the technology, yes it’s coming, the technology is coming but I don’t think anyone probably knows exactly how long it’s going to take. If it’s going to be one year, three years, five years, but all of this is definitely coming together and then you service the next one, which obviously is creating the experience from the little parts that exist now and we are really happy to be the hands of that experience.

David Kruse: Yes, and you guys have put yourselves in a great position, because I mean you have been working on this issue for soo many years and you have a commercial product and I know VR is a new space, but still that’s exciting for you guys. And I am curious how – with the virtual reality, how does that, the headset, how do you track your hands. What sensor, technology are you using to make that happen?

Michael Buckwald: Yes, that’s a great question. So Leap in general, the way we track the hands is through really – its 99% software that sits on either the host computer on the PCA or if it’s a mobile gear headset, sort of arm processor that’s inside the device and our software runs very, very fast so we can run it hundreds of frames a second, even off of a very light weight processor, like a mobile processor and still have a great experience with very little latency and high fidelity tracking. There is hard, but it’s very, very and expensive mobile hardware, so the actual sensors that are in the sort of luminal immersion peripheral that may people know about or even the modules that would be imbedded inside the area and space of it, which is the main business model for VR so that everyone gets it and its inside the headset. Both of those are essentially the same resolution, VGA, off the shelf sensors. It’s an off the shelf ID, is that it cost just a few dollars to embed and you know that’s – one of the things that’s great about that is that anything that can have a camera can fit it. So in a world where every zone has in some cases certainly one very high resolution camera but sometimes many cameras, really there is nothing that our device can’t fit into already and the cost is very low and gets lower all the time without investment for us. But the best thing is that we can push huge updates as we have done every year for really the past three years at least. That massively changed the performance of the tracking and the technology for different [inaudible]. So for example there was version 1 three years ago, which sort of just started with our fingertips and it was still the best tracking in the world at the time and it was the first tracking, and even now potentially was the first time anyone had tracked 10 figures, but they were joint stuff. So a year later we pushed that version 2, which added the joints, but was not built for VR and then version 3 Orion, which is what you were talking about, which we push that now about six months probably, about five or six months ago that have been, due to the same level of joints and fidelity and such that for VR specifically and solve some of the problems that are unique to VR when its mounted on the headset, which we think is the right place where it could be, because if it’s on the table in front of you, you have to worry about, are you in the field of view or not, but if its mounted on the head, then our field of view is bigger than any of the H1B field reviews by far. So anything you can see and hear, you can interact with your hands, which makes a lot of sense and that’s kind of how we interact with the real world where we either turn and look at something before we pick it up. So that’s kind of how we have been thinking about that VR and physical device. But the software is great and the push up, its, it’s been great for us more.

David Kruse: Interesting, okay.

Michael Buckwald: It’s the same, basically the exact same device that people have to get in their closets from three years ago. I think it will take out down with the new Orion software and it will be a completely different app as well as on you…

David Kruse: Wow, okay and I think we have time maybe for one last question.

Michael Buckwald: Yes, one more will be great.

David Kruse: And so I’m kind of curious, where you want to take Leap over the next five years with AR and VR? What’s kind of your ideal vision if things could play out for your technology?

Michael Buckwald: Yes, it’s a great question. I think that we – we definitely want to see it embedded into all sort of VR area, that’s what we want the version of our tracking to be. Consider the fact that a minimum part of the experience and by virtual of that we hope that we can help push that connection between people and technology and we could have kids growing up, which is just fundamentally different conception of the world and knowledge based on that different connection, that are bolted with the connections and with the technology because that has it and certainly for AR we want our technology embedded in all the area and similarly we want the thing that’s enabling. It’s not the sort of augmented reality passive TV show or movie that pleases you and makes you a creative magical forest where you can’t touch anything or interact with anything and instead it would be a magical forest that you live in and can interact with them. It looks like the real world and that’s to have the other hand there that really in some ways is responsible for the majority of the transformative parts of that. Because we are used to looking at movies of things, pictures of things, but – and they can be very compelling. We could have like an IMAX screen with 3D or something, but the ability to walk out and touch things. That sort of really makes the feeling actually sharable, the rest of your life as if your walking on the streets of Paris. So that’s having the hands really part of that.

David Kruse: And from a technology standpoint, are you – is there still a lot more development to reach there. I mean I know from the commercial standpoint you don’t have to integrate with the headsets and stuff, but technology device is on the Orion. It sounds like you are pretty close to…

Michael Buckwald: Orion is pretty close. Certainly it’s not perfect yet and it’s in data for that reason. I think we have a few – well there is a few – there is certainly some number of lots of work left before we would think of it as production worthy, but there is also always a bunch of you know little bars and improvements and in addition to these sort of annual big updates we should also pursue other things that improve, some amount of the tracking and well things here and there in different ways dynamically, which is another advantage of it not being software so if it were.

David Kruse: Makes sense, okay. Well, I think that just about does it. So Michael, I definitely really appreciate you coming on and telling me about your background and what you guys are doing in motion, it’s really interesting. You guys have often paved the way that makes AR and VR actually interactive and interesting in and of itself. So I appreciate what you guys are doing.

Michael Buckwald: We are having fun here and it’s an honor to be in a place where we can sort of see everything that’s happening with VR and also everything that’s happening AR that we pretty much want to make it a reality, so yes it’s been great talking to you, and yes, thanks again.

David Kruse: Yes, thank you, and thanks to everyone for listing to another episode of Flyover Labs and thank you Michael again. We appreciate it.

Michael Buckwald: Great.

David Kruse: Buy everyone.