IBM®
Skip to main content
    Israel [change]    Terms of use
 
 
 
    Home    Products    Services & solutions    Support & downloads    My account    
IBM Research

Coverage Directed test Generation

Functional Coverage


Overview

Client

IBM Server Group.

Challenge

One of the main challenges of Simulation-based Functional Verification is closing a feedback loop from the domain of covered events back to a generator that produces new stimuli to the tested design. Coverage Directed test Generation (CDG) is automated feedback, which can reduce the manual work in the verification process and increase its efficiency.

 
Coverage Directed test Generation before using Machine Learning


Solution

The objective of CDG is to automatically provide directives that are based on coverage analysis to the test generator. This can be further divided into two sub-goals:
  1. Provide directives to the test generator, which help in reaching hard cases, namely non-covered or rarely covered tasks. Achieving this sub-goal can shorten the time needed to fulfill the test plan and reduce the number of manually written directives.
  2. Provide directives designed to reach any coverage task, using different sets of directives when possible. Achieving this sub-goal makes the verification process more robust, since it increases the number of times a task is covered during verification. Moreover, if a coverage task is reached via different trajectories, the chances of discovering hidden bugs related to this task increase.

CDG using Bayesian Networks
The inherent uncertainty embedded in the CDG setup implies casting this problem in a statistical inference framework. Bayesian networks offer an efficient modeling scheme by providing a compact representation of the complex (possibly stochastic) relationships among the CDG ingredients, together with the possibility of encoding essential domain knowledge. The figure below presents a flow chart of the CDG process and the role of the Bayesian network that closes the feedback loop.

The Bayesian network models the world using:
  • A set of interface nodes that link the Bayesian network, on one hand, to the coverage report, and, on the other hand, to the Random Test Generator (RTG)
  • A possibly additional set of hidden nodes, which model internal ingredients (hidden causes), that may affect the covering task
  • A graph structure that specifies the relations between the nodes

While the first item represents the observations directly accessible (to some degree) to the Bayesian network, the last two items represent a means for domain knowledge encoding.

CDG Via Bayesian Networks Procedure
The CDG via Bayesian networks procedure starts with a learning phase in which the verification engineer first specifies the model (nodes + graph structure) and then collects a training set, composed of RTG's directives and corresponding coverage reports (simulation outcomes). All this information is used to train the net's parameters and possibly its structure.

Once learning is completed, the Bayesian network is used to generate directives aimed at reaching the desired coverage events (for example, non-covered events). Coverage events are entered to the net as evidence. Using statistical inference procedures, the net produces directives to the RTG (values or marginal distributions over the relevant interface nodes), aimed at reaching the desired coverage events.

We do not model the behavior of the design, which is typically a large and complicated (deterministic) finite state machine. Rather, we model the CDG process itself, namely, the trial-and-error procedure governed by the verification team, which controls the test generation on one end and traces the progress of covering the test plan on the other. This process requires the ability to make decisions under conditions of uncertainty, especially for hard-to-reach coverage tasks. A Bayesian network trained on the already reached coverage tasks and their respective stimuli, will reveal (possibly hidden) causes and suggest explanations and stochastic production rules, which in turn can be used to redirect the test generation process and provide enhanced coverage.

 
Coverage Directed test Generation after using Machine Learning


Benefits

  • High quality coverage
  • Extremely efficient coverage rate
These benefits mean a faster, more complete coverage process.

 
 

 


    About IBMPrivacyContact