SALT

Specification & Abstraction Language for Testing (SALT)

Concepts and background:

Current practice of test generation for function testing of Application Programming Interfaces (APIs) involves the following steps performed manually:

Read the informal specifications for functions under test.
Develop the set of test variations that would "adequately" cover the test requirements.
Convert these variations into executable test cases.

There are several drawbacks in these steps due to their manual nature.

It is less efficient.
It does not lead to measurable test coverage.
It does not promote effective reuse and cooperation among testers.

To automate the current process, we have proposed a new sequence of steps. These steps are:

Capture testers' mental model (State/context variables for the component under test, logical Input/Output partitions for each function under test, result relationships among these input/output partitions including updates to context variables) using SALT (Specification and Abstraction Language for Testing), a language developed at Research.
Compile the test model to perform consistency and completeness checks
Use Heuristics for combinatorial optimization of input parameters in generated test variations for each function based on fault models for boolean expressions.
Use Algorithms based on Dataflow dependency analyses to combine test variations from different functions into a test sequence.
Minimize the number of test sequences using the data values for logical partitions.
Generate executable Test case using the tester specified code template fragments.

Characteristics of APIs which are suited for application of current SALT technology:

Set up of the input and environment variables is simple and does not require invocation of a sequence of APIs under test.
The relationship between Context, Input, Environment, and
Output variables of the API is simple. For example, the context variables are assumed to be independent of each other and are linear functions of input/ environment variables.
Concrete data values associated with the logical partitions have finite cardinality.
Expected output can be verified through comparison with actual
output explicitly within the test case itself (Test cases for verification
of asynchronous events, timed response etc. cannot be generated
automatically).
Test cases corresponding to the APIs under test can be
represented in a "Template" format that can be specified by the
tester. Complete test case can be generated through appropriate
substitution of input/environment/output variables in this template.

Inputs to SALT compiler:

Currently, the following information is captured by the user in the SALT language in the form of a text file created with any text editor.

Logical Partitions of all input variables for API under Test.
Logical Partitions of all Environment variables affecting the API
under test.
Logical Partitions of all Output variables produced by API under
test.
Context variables to "remember" execution history of API invocations.
Relationship among context variables; output, environment, and input partitions expressed as Boolean conditions.
Updates to context variables as a result of function invocations.
Data values corresponding to the logical partitions to be used in test cases.
In addition, the user also gives a template for the test cases to be
generated.

Outputs from SALT Compiler:

List of Test Variations which cover the relationships specified in 5.
above.
A list of test variations which cover sequences of function invocations.
Actual test cases corresponding to these test variations

For more information on SALT, please read "SALT - An Integrated Environment to Automate Generation of Function Tests for APIs" by Amit Paradkar.