Canon Mixed Reality System (MR System)

Canon presented a new augmented reality project they developed on a recent Virtual Reality Expo (IVR) in Japan. Named Mixed Reality System (or MR System), the image data processing technology offers integration of real and virtual worlds in a different way than their previous augmented reality (AR) systems, since the new system enables you to interact with computer generated (CG) elements.

AR is finding broader use in gaming, advertising and other applications, simply by superimposing CG images onto the real world. Canon’s MR System takes this process a step further by allowing multiple users to interact with full-scale 3D CG images from any point of view. So, how does it work?

Two video cameras located on the head-mounted display (HMD), capture the video from the real world and send it to a computer. Since the cameras mimic user’s left and right eye, the image they record represents user’s stereoscopic view. Computer combines image processing and positioning of the user to form an image of the 3D CG element and places it in virtual space which is superimposed with the real world image.

Canon developed proprietary registration markers for the MR System that enable proper CG positioning within real-world environments. Once a marker is captured by the HMD’s video cameras, the system adjusts the layout of the virtual image according to the relative position of the HMD. The system also supports optical- and magnetic-type sensors which can be used to enhance the interaction with virtual objects.

Displayed CG visuals are aligned with the HMD’s position and orientation one frame at a time, thus enabling integrated video that instantaneously reacts to changes in a user’s position or movement.

CG visuals in the computer and the real-world environment are combined with high precision and displayed on the small monitors located inside of the HMD. The system uses a free-form prism with a unique three-sided configuration which allows the MR System to produce images with low distortion in peripheral areas caused by optical aberrations.

Even though the images are delivered through a video feed, by matching the video cameras’ optical axes with those of the light entering the eyes from the small displays, the system delivers a somewhat realistic experience, and creates the impression that visuals are being seen with the naked eye.

Canon plans to pitch their product for applications in industrial design and other design fields. While these areas do use a lot of 3D data, I believe they should have adapted their system to be tested on gamers, who are known as early adopters of new technologies.

During the design phase, the system can be used for collaboration between designers, thus reducing the number of prototypes and shortening the amount of time spent on development.

The company will offer customized systems based on customer needs. For example, the system can be used to produce simulations of the optimal layout of production equipment at a manufacturing site or in home decoration before it is done, or it can be used for product testing where a virtual product is presented to testing groups.

The folks from Canon haven’t announced precised pricing rates, since they will vary depending on system configurations in accordance with usage requirements. Mixed Reality System will be available in late July 2012.