Modern verification is a highly automated process that involves many tools and subsystems. These tools produce a large amount of data that is essential for understanding the state and quality of the design under test and the progress of the verification process. The complexity of the verification, the amount of data it produces, and the complex relations between the data sources demand sophisticated data science techniques.
- These techniques can be divided into three high-level categories:
- Descriptive and diagnostic: Analytics and visualizations that allow users to gain insights from the huge amounts of data collected and thereby better understand the behavior of tools and environments. For example, a tool that can answer the question: Did my tests cover all aspects of the tested design?
- Predictive: Based on the past, build learning models and AI-based systems that try to predict what is most likely to happen and identify problems before they occur. An example of this would be a tool that can predict if a change that was made recently will cause failures in specific areas.
- Prescriptive: A set of tools and techniques to help improve processes and their outcomes, such as an AI-based system that can replace the manual creation of tests to cover hard-to-reach areas of the system.
Our arsenal of hardware quality technologies tools include the following:
Machine learning quality and testing
With the vast use of AI systems these days, there is an increasingly high demand for quality assurance tools and methodologies. Our group performs world-leading research and provides cutting-edge technology for analyzing and validating the quality of ML-based systems.
- Our technology for ML quality testing is called FreaAI. Its capabilities include:
- Model drift detection: Use weak data slice technology to diagnose ML model changes and drifts.
- Governance and regulations: Verify compliance with government regulations for data privacy and security in ML models.
- combinatorial testing: Bridge the gap between ML solutions and their business requirements using combinatorial testing.
- Statistical testing: Use statistical and density-based analytics to identify weaknesses of ML models.