The Analytics & Quality Technologies group researches methods and solutions to improve both hardware quality and machine learning (AI). Hardware quality assurance has always been and continues to be one of the most challenging and time-consuming activities in the modern microprocessor design process. Machine learning (AI) techniques are the key for many complex daily tasks and challenges – for hardware quality and in general.
- Our team broadly addresses these two tasks. The group’s products consist of state-of-the-art analytics and ML-based software to guarantee the highest quality for IBM tools and infrastructures. The group is composed of two teams of experts that focus on the following areas:
- Hardware quality - Provide a broad set of software solutions and apply data analytics techniques (ML and others) to track and optimize large-scale hardware verification processes.
- Machine learning quality and testing - Develop end-to-end methodologies and a framework to identify, predict, and send alerts about malfunctions or weaknesses in ML-based systems.
Wesam Ibraheem,
Manager, Analytics & Quality Technologies,
IBM Research - Haifa
Research
Hardware quality
Modern verification is a highly automated process that involves many tools and subsystems. These tools produce a large amount of data that is essential for understanding the state and quality of the design under test and the progress of the verification process. The complexity of the verification, the amount of data it produces, and the complex relations between the data sources demand sophisticated data science techniques.
- These techniques can be divided into three high-level categories:
- Descriptive and diagnostic: Analytics and visualizations that allow users to gain insights from the huge amounts of data collected and thereby better understand the behavior of tools and environments. For example, a tool that can answer the question: Did my tests cover all aspects of the tested design?
- Predictive: Based on the past, build learning models and AI-based systems that try to predict what is most likely to happen and identify problems before they occur. An example of this would be a tool that can predict if a change that was made recently will cause failures in specific areas.
- Prescriptive: A set of tools and techniques to help improve processes and their outcomes, such as an AI-based system that can replace the manual creation of tests to cover hard-to-reach areas of the system.
Our arsenal of hardware quality technologies tools include the following:
Machine learning quality and testing
With the vast use of AI systems these days, there is an increasingly high demand for quality assurance tools and methodologies. Our group performs world-leading research and provides cutting-edge technology for analyzing and validating the quality of ML-based systems.
- Our technology for ML quality testing is called FreaAI. Its capabilities include:
- Model drift detection: Use weak data slice technology to diagnose ML model changes and drifts.
- Governance and regulations: Verify compliance with government regulations for data privacy and security in ML models.
- combinatorial testing: Bridge the gap between ML solutions and their business requirements using combinatorial testing.
- Statistical testing: Use statistical and density-based analytics to identify weaknesses of ML models.