Machine learning systems help technology simplify many complex daily tasks and challenges. But the performance of machine learning models can deteriorate over time. This "drift" results from several factors, such as changes in the input data and differences in the context and deployment environment. Our team is developing end-to-end methodologies that include new ML technologies to create a framework that can identify, predict, and send alerts for drift in ML deployment.
Our ML deployment drift identification technology allows users to "slice" ML model data and measure model performance over the different slices, similar to how CT scans create and allow physicians to review image slices of the body to identify problems. Our solution scans the trained ML model and its hold out set of test data. As part of the work in this area, we're carrying out research at the core of machine learning to make advances in the very difficult problem of distribution identification.
Successful implementations of the technology can help us build more robust and accurate business-grade machine-learning solutions.