People are buzzing about the accomplishments of Yost Engineering, Inc. (YEI), a group that recently revealed its innovative use of Epic’s Unreal Development Kit (UDK) to demonstrate real-time, markerless full-body motion capture as well as head-tracking virtual reality (VR) techniques in richly detailed Unreal Engine 3 (UE3) test environments.
Watch the work-in-progress video:
Epic spoke with Paul Yost, chief of research and development, Yost Engineering, to obtain a better understanding of how his team developed the compelling mocap and VR demonstrations.
“In this demo, we've affixed the YEI 3-Space Sensor to a Sony HMZ-T1 head-mounted display and used 16 additional 3-Space wireless sensors to mocap the performer. The YEI 3-Space Sensor is performing all head and body tracking in real-time and providing the head and body segment orientation outputs as a set of quaternions,” he said.
“The head-orientation quaternion is then directly used to drive the camera in UDK. Thus, taking advantage of UDK’s ability to bind external DLLs, getting orientation data into UE3 is only a few lines of UnrealScript to achieve head tracking using a YEI 3-Space Sensor.
“Likewise, to control the avatar within the virtual world, we're using orientation data from 16 additional performer-mounted 3-Space sensors to directly drive the bone orientations on the rigged character model. Again, in UE3 this requires a very small amount of code to achieve a very natural-feeling immersive virtual experience.
“With the integration of RealD into UDK the scene can be rendered in 3D on the headset without any additional external third-party applications. The 3-Space Sensor is small enough to be attached to a variety of head mounted displays or just attached to the head to add immersion to a fixed screen experience. And by augmenting the head-mounted display with our 3-Space wireless sensors, suiting-up for a fully immersive VR experience only takes minutes.”
The YEI 3-Space Sensor is a high-accuracy yet low-cost orientation sensor and inertial measurement unit (IMU. It uses a three-axis gyroscope, a three-axis accelerometer, and a three-axis magnetometer, along with advanced on-board error-compensation and Kalman filter-based sensor fusion algorithms that allow the unit to easily output ready-to-use, real-time orientation data and other inertial data with very little effort.
Established in 1999, YEI Technology is a division of Yost Engineering that performs technology-related research and development that results in commercial products and IP.
Yost’s team has been working on the YEI 3-Space Sensor for years, and given its broad range of potential uses – including but not limited to medical simulations, military and aerospace applications and entertainment software – his team has developed multiple versions, such as embedded, wireless and data-logging sensors.
“We’ve been using our wireless versions of the sensor for markerless motion capture so we can record real-time movement without suits, cameras or special studios,” said Yost. “Needless to say, we're really excited about the opportunities for the sensor and are just getting started exploring them all with UDK.”