Computer Vision and Augmented Reality

Technology and solutions for Object Recognition and Augmented Reality applications on cloud and mobile platforms

Our group advances technology for object recognition, deep learning, and 3D navigation, and harnesses it to build novel industrial and commercial solutions. Our portfolio includes AR applications providing Visual Operation Guidance for industry workers, Object Detection and Recognition employed in marketing and retail, and more. Our solutions are deployed as cloud services and mobile applications. Beyond the technical scope, our mission is to extend and expand people’s vision and comprehension of the environment.



Localization and tracking in 3D environments

We develop technology for Augmented Reality applications, which include 3D object mapping, recognition, and tracking. Our methods enable working with a wide range of scenes without resorting to texture-rich markers for easier localization. The applications are adapted to and deployed on mobile platforms, such as smartphones and AR glasses.

Object detection and recognition

We explore a number of research directions in the Deep Learning and Computer Vision domains. We use both classical methods involving probabilistic models and manually engineered descriptors, as well as state-of-the-art deep learning technologies. Our solution features robust detection of objects in the wild and the ability to learn one-shot custom object detectors by manually marking just one or a few examples of each object. In addition, we handle the problems of hand pose and gesture recognition, action recognition, semantic segmentation, scene recognition, and articulated joint and fiducial point detection.



Visual Operation Guidance

Visual Operation Guidance is a platform for providing AR-based assistance to technical workers in the field and to consumers of technical equipment. It enables designing applications for self-guidance in the environment for which AR content was prepared, and peer guidance for AR-based remote assistance in a random situation. AR-based guidance increases the efficiency of a technician’s work, the percentage of issues resolved from the first visit, and reduces the number of errors stemming from encountering unfamiliar equipment.

Object recognition

CVAR has multiple deep learning-based projects, some of which are implemented as cloud services. These include detection of objects, body parts, and joints; ground cover classification; and more. Services like logo detection are of commercial value for companies performing market study for their products.

Solutions for retail business

We offer services for automatization of planogram compliance and enforcement of Product License Agreements in retail stores.

Multi-modal context-aware conversation engine

This intelligent conversation engine receives user queries (via speech), detected objects and finger pointing, IoT telemetry, while maintaining the context of the conversation.


Oded Dubovsky, Manager Computer Vision and Augmented Reality, IBM Research - Haifa