Sony SmartAR – Integrated Augmented Reality technology

Sony has developed integrated Augmented Reality (AR) technology which enables additional information to be displayed on the device’s screens such as virtual objects, or images and texts that cannot be identified by visual perception alone. Smart AR uses the markerless approach, and enables quick recognition of objects and can be tracked at high-speed along with the movement of the camera.

SmartAR technology combines object recognition technology (markerless approach in which no special markers are required) for recognition of general objects such as photographs and posters with Sony’s 3D space recognition technology, which has been developed through the research of robots such as AIBO and QRIO. The technology is also capable to display the AR information over an extended space.

AR information can be displayed on the captured image which appears on a device’s screen, including those images that do not have any special markers for AR. The object recognition technology identifies objects by analyzing features detected from a portion of the image together with their positional relationship. A probabilistic method that matches local features with minimal calculations enables high-speed recognition that is resistant to changes in lighting or the position of the object. In addition, recognition is still possible even if the object captured appears to be comparatively small in the display.

The natural-feeling tracking of the camera’s movements at high-speed has been realized by combining object recognition technology with matching technology that uses features detected from a portion of the image and image tracking technology that is capable of dealing with changes in the shape of the object.

Even if the AR image is a large virtual character whose size exceeds the dimensions of the device’s screen, the technology allows the user to grasp the whole virtual character when the camera is moved around. Furthermore, it is also possible to move the virtual object in the actual 3D space as if it was really there.

3D space recognition technology is based on use of the difference observed by the camera movements to estimate the shape of the 3D space and the position and angle of the camera. By combining this with object recognition technology, devices become capable to identify and remember 3D space constructions. Information can be easily acquired and navigated simply by directly touching the AR information displayed on the smartphone or device’s screen.