Our team is currently developing a solution called Cognitive Radiology Assistant - the next generation of cognitive assistant for radiologists. The system analyzes medical images and then combines this insight with information from the patient's medical records to offer clinicians and radiologists support for decision-making. By applying advanced reasoning and visual technologies, Cognitive Radiology Assistant filters out the most relevant images that point out abnormalities and provides insight into medical findings.

Today, emergency room radiologists may have to examine as many as 200 cases a day, with some studies containing as many as 3000 images. Each patient's imaging studies could be around 250GB of data making an institution's image collection petabytes in size. This is where Cognitive Radiology Assistant steps in and contributes considerably to reducing diagnosis time and clinician fatigue. The need for Big Data analytics to help sift through all this medical data is clear.

Cognitive Radiology Assistant - Addressing the challenge of breast cancer

Worldwide, breast cancer comprises close to 30% of all diagnosed cancers in women, making breast cancer the second leading cause of death for women. Accordingly, a great deal of work has been performed to facilitate breast cancer detection—including advanced computer-aided diagnosis tools for mammography images. However, until now, little or no work has been done in the area of the advanced decision support tools that combine multi-modal image analytics and clinical data analysis.

From the technical point of view, the research includes advanced computer vision techniques to allow automatic extraction of diagnostically relevant features. Machine learning tools are also used to combine multimodal semantic image descriptions (for Mammography, Ultrasound and MRI) with clinical data, facilitating estimation of correct differential diagnosis and patient management recommendation.

Project Recognition / Visibility

This video presents IBM's Eyes of Watson demo for breast cancer detection, which was presented at the 2016 Annual Meeting of the Radiological Society of North America (RSNA). The demo highlights IBM's capabilities in the area of medical imaging with a question-answering format. The Watson-based technology is designed to serve as a cognitive assistant to radiologists in their workflows.

Working to Make Healthcare Smarter Every Day

A TV ad showcasing Cognitive Radiology Assistant technology as the future for Watson for Imaging have received good reviews with over nearly half a million.

  • TedMed Aya's Soffer presentation
  • Cognitive Radiology Assistant was demonstrated at CEBIT 2015 in Hanover, Germany in the Watson Booth leading to local TV and press coverage
  • Internal IBM award "Outstanding Science Accomplishment" 2014


Flora Gilboa-Solomon, Manager Medical Imaging Analytics, IBM Research - Haifa

Efrat Hexter, Manager Medical Imaging Solutions, IBM Research - Haifa