Hands-on: Microsoft’s “Mixed Reality” VR Motion Controllers

Microsoft debuted the lengthily-named “Windows Mixed Reality motion controllers” back in May, but until now we haven’t had a chance to actually try them out. During a recent meeting with Microsoft in San Francisco, I got to try the VR controllers for the first time paired with the Acer Windows VR headset.

Microsoft’s VR controllers are designed to let you reach into VR and interact naturally with the virtual world. With both a trackpad and a thumbstick, they look like a crossbreed of the Oculus Touch controllers and the Vive controllers.

In addition to the trackpad and thumbstick, there’s also a menu button and a Start button, as well as a grip button along the handle. The big circular parts on the front contain an array of LEDs which provide bright markers for the headset’s on-board cameras to detect and track. Microsoft tells us that the shipping version of the controllers will indeed use visible-light, just like we’ve seen in renderings and promo videos. (Microsoft didn’t allow any pictures of the controllers during my hands-on time).

Buttons and Inputs

Image courtesy Microsoft

Grabbing the controllers for the first time, they didn’t feel quite as elegant as either Touch or the Vive controllers. The odd side-by-side trackpad & thumbstick arrangement is useable, but seems to effectively put neither of the two in an ideal position for your thumb. The grip button is indeed a binary button (rather than being pressure sensitive), and doesn’t feel so much like a “grab” as it does a clicky button press with your palm.


Though they resemble Touch with their ring-shaped tracking appendages, the Windows motion controllers are actually noticeably larger and clunkier thanks to the placement of the tracking rings, which don’t encompass your hand like Touch, making the controllers easier to bump together, especially when their physical outline is hidden in VR.

The shape of the rings is necessary though, as they need to present a substantial surface area from which the headset’s on-board cameras can track their movement. Though I was using the Acer dev kit headset, our understanding is that these controllers will work with any of the soon to be released Windows VR headsets (all of which feature on-board cameras).

Pros and Cons of Inside-out Controller Tracking

Image courtesy Microsoft

This method of controller tracking differs from both the Rift and the Vive in that it’s the cameras on the headset which are watching the controllers to track their movement (which is called ‘Inside Out’), whereas the Rift and Vive both use external sensors to track their controllers (called ‘Outside In’).

The upside to this approach is that you don’t need to set up any external trackers, but the downside is that the controllers must always be in view of the headset’s front-facing cameras to be properly tracked. Thankfully, the size of the tracking volume felt reasonable; for basic use (like reaching out in front of me to grab virtual objects), I didn’t feel like my reach was artificially limited by the camera’s field of view.

Outside of the Box Tracking

And for times when your hands will go out of the camera’s field of view, Microsoft is doing its best to compensate. When that happens, the system relies purely on the controller’s on-board IMU to estimate positional movement until it reappears in the camera’s view. This works well enough for quick jumps in and out of the camera’s view, but after a second or two, the IMU-only tracking estimation is too unreliable, and it appears that the system will eventually freeze the location of the controllers in the air and only feed them the rotation data from the IMU, though they snap quickly back into their proper place as soon as they’re brought back into view. It remains to be seen how much this limitation (the need to be seen by the front-facing cameras) will impact different VR games and apps, and how effectively it can be designed around.

Tracking Accuracy

As for the tracking accuracy when they are in sight of the camera, I did see some jumpiness here and there—especially if I was rotating my body while moving the controllers—but on the whole they seem entirely usable, and (in my short time with them thus far) to be more accurate than the PlayStation Move controllers.

As part of my testing, I played Arizona Sunshine (2016), and found that guns were steady when I held them out in front of me and aimed down their sites; I didn’t have any trouble landing zombie headshots. Granted, for inside-out controller tracking, holding a gun up in front of me to aim is pretty much the best case scenario—I’m curious to see how other common input modalities hold up (like shooting a bow and arrow or swinging a sword).

The occasional jumps didn’t present much issue in a shooting scenarios, but for more precise uses, like VR drawing, painting, and animating, it remains to be seen if those jumps will cause any usability issues.

– – — – –

Microsoft says that the Mixed Reality motion controllers will be bundled with Windows VR headsets starting at $400 this holiday.

SOURCE: https://www.roadtovr.com/microsoft-mixed-reality-vr-motion-controllers-hands-on/

You can finally stuff your head into a Windows VR headset

After a few months of waiting, you can snap up a Windows Mixed Reality headset for yourself… if you meet the right conditions, that is. Microsoft is now selling both the Acer and HP Developer Edition headsets at respective prices of $299 and $329, but only to developers — you can’t pick one up just because you think an Oculus Rift is too expensive, unfortunately. The HP model is also out of stock as of this writing, so you can’t be too picky.

Thankfully, you’re largely getting the same experience. Both wearables include a pair of 1,440 x 1,440 displays, a 95-degree field of view, support for 90Hz refresh rates (the usual target for VR) and a single cable that carries both HDMI video and USB data. Those aren’t mind-blowing figures, but that’s not the point. This is more about fostering VR and AR apps to make sure there are plenty of them when Windows Mixed Reality hardware is available to the general public. If all goes well, Microsoft will have laid the groundwork for taking VR and related technologies into the mainstream.

SOURCE: https://www.engadget.com/2017/08/01/windows-mixed-reality-headsets-available/

A Look at the World’s First Interactive Mixed Reality TV Show

The Norwegian program uses MR to bring competition to life; letting viewers play along.

It was only a matter of time until mixed reality made its way to broadcast television. Developed by The Future Group and FreemantleMedia (the company responsible for American Idol and X Factor), Lost In Time is a Norwegian game show that uses interactive mixed reality technology combined with a customized green screen studio to transport contestants to different landscapes scattered across time and space.

From obstacles in the Jurassic Era, puzzles in the Roaring 20’s, and challenges in Medieval times, contestants will shoot, drive, navigate and solve their way to glory.

24 different challenges fall into four primary game categories with each competition lasting 90 seconds each. Physical props are spread throughout the green screen studio to help guide participants. Otherwise the rest of the scenery is rendered digitally using Unreal Engine 4 in post.

This explains the beautifully rendered visuals (this level of quality requires extensive rendering time) as well as the lack of a live broadcast. Participants competing will not be wearing any form of a virtual reality headset, which means only spectators will have a chance to view the full mixed reality experience.

According to Bård Anders Kasin, co-Founder of The Future Group, this was an important strategic move:

“We give them enough of that environment so that they can compete that challenge, and we use green screen to transfer them into the virtual world,” said Kasin during an interview with GameCrate. “Of course if they are driving something, they see the environment like you would on any motion simulator platform today. But we’re not using VR goggles, because that’s very important in terms of seeing their emotions, we need to see how they’re reacting and all of that.”

However the Lost In Time experience extends much further than the mixed reality competition. Viewers at home also have the chance to get their hands dirty by competing in touchscreen mobile versions of the full challenges. Instead of suiting up and physically running through each challenge, players use their smartphones to touch and swipe their way to enticing cash prizes.

“The way we’ve built it is that you’re doing the same challenges at home as the people in the studio, and that of course is a very tricky thing to do,” continued Kasin. “But we’ve come up with various ways of solving that. Let’s say that you’re driving or flying something. You’re flying a plane in 1920s New York, which is one of the time eras we go to. In the TV studio we basically place the contestant on a motion platform simulator, similar to a flight simulator, but of course the virtual technology allows us to do a full multi-camera production so it appears as if they are flying an actual plane inside that virtual world. Then on my mobile phone I would be flying my own plane, and doing exactly the same challenge, basically steering it with my fingers, swiping left and right to steer it.”

“We have ported those challenges to mobile,” spoke Ellen Lyse Einarson, the company’s Director of Games, to GameCrate. “So the viewers at home can do those same challenges, but we’ve also added meta-features to ensure user retention, so that people also play throughout the week, and earn virtual currencies in order to participate in the biggest tournaments with the bigger physical prizes when the show is live. A player at home will be chosen to win the same money reward that one of the contestants takes away, so it’s obviously in their benefit that the contestants do really well.”

This is just another clear indicator of the endless entertainment potential mixed reality can provide. Immersive technology could revolutionize how TV and film projects are developed from the ground up. At least this is the hope of The Future Group as they continue to dive deeper into MR. The program is only half way through its 8 episode season in Norway, with each episode lasting a total 43 minutes each. Hopefully U.S. broadcasters smarten up and jump on this goldmine as soon as possible.

SOURCE: https://vrscout.com/news/interactive-mixed-reality-tv-show-lost-in-time/


With passion for immersive technologies we aim to provide you with everything you need to start and grow as a developer. We also provide custom application development for businesses. Contact us via the form below and one of our staff will be in touch with you shortly.