Quality as a Service
Software quality and reliability are essential for attaining a high degree of user satisfaction and acceptance of software products. As software systems and applications become increasingly more complex, the limitations of existing testing and performance analysis practices becomes very challenging. The Quality as a Service group aims at making significant contributions in the area of software quality by developing new technologies, practices, and tools, working in close collaboration with development, testing, and performance analysis groups.
Testing and Coverage Analysis
- IBM Functional Coverage Unified Solution (IBM FOCUS)
IBM Functional Coverage Unified Solution (IBM FOCUS) is an advanced test planning tool for improving the testing of an application. IBM FOCUS uses combinatorial test design (CTD), also known as combinatorial testing, to generate an efficient test plan that provides consistent coverage across the test space at a known depth, while significantly reducing the required resources. IBM FOCUS is independent of an application's domain and can be applied at different levels of testing. IBM FOCUS can also read existing tests, analyze their functional coverage, select a subset of the tests that maintains the same coverage, and generate new tests to close the coverage gaps. IBM FOCUS requires a user definition of the test space and provides advanced review and debugging capabilities to verify that the test space was defined correctly.
The FOCUS technology is extensively used by software testers across IBM and its external clients and partners, and has contributed to realized revenue and development savings of hundreds of millions of dollars.
More information on IBM FOCUS is available here.
A recent blog post on CTD in IBM FOCUS is available here.
The SPARQ Group is highly active and leading worldwide research on combinatorial testing. A full list of our published papers, in top-quality venues, is available here. We are also leading and co-organizing the International Workshop on Combinatorial Testing (IWCT), being held annually in conjunction with the International Conference on Software Testing (ICST).
Contact Person: Rachel Tzoref.
- IBM Code Coverage Unified Solution (IBM CoCuS)
The IBM CoCuS tool provides extensive views and analyses for code coverage. IBM CoCuS reads code coverage data generated by different tools such as EMMA, IBM Binary Prober, and Gcov. CoCuS can be easily extended through an open Java API to read code coverage data from other tools. CoCuS provides traditional coverage views such as source views and hierarchical views, and more advanced analyses that imply, given code coverage data, what functionality is missing from a test suite and what functionality is covered. Additional analyses compare the coverage data produced by different sources and enable understanding differences in an application's behavior across different sites or test suites. All coverage views and reports are exportable to html format.
Contact Person: Orna Raz.
- HLAM (High Level Application Model)
HLAM is a framework for generating user defined loads on the machine/OS resources (CPU, Memory, I/O, Networking …). A user can specify utilization scenarios using the JSON format. HLAM calibrates itself on the target system and then automatically generates a synthetic workload that follows the required utilization scenario. HLAM can be used for: stress testing, integration testing, problem recreation, and performance comparison between different systems. It currently supports Linux and AIX OS.
Contact Person: Sergey Novikov
Trace and Log Analysis
Easer helps humans in exploring various logs and traces. It works with any semi-structured data – data that has multiple entries over which an analysis should be run, and is in ascii (including html) format. The tool supports positional searches, non-positional (free) text searches, and customized searches. The tool has a UI that allows a user to specify positional and free-text searches without requiring any programming skills.
Contact Person: Orna Raz
- Trace Analyzer
Trace Analyzer is a framework for collecting, analyzing, and visualizing performance data from OS and/or user-space sources. Its main purpose is to allow a user to identify performance bottlenecks. Trace Analyzer is a modular tool. It allows computing various metrics on a trace and its parts. It also enables building custom graphical views, such as different types of charts; combining different data together into the same graph; or showing different aspects of data in separate charts from the same trace. Trace Analyzer supports a variety of tools in different environments; generic Linux tools, such as tcpdump and sar, custom tools such as pthread_mon, AIX and Linux tools such as nMON, NAS monitoring, and many more.
Contact Person: Marcel Zalmanovici
Methodologies for Software Quality
- System Grokking Technology
System Grokking Technology is a software architect assistance technology that is designed to support incremental and iterative user driven understanding, validation and evolution of complex software systems through higher levels of abstraction. It enables semi-automatic discovery, manipulation and visualization of groups of domain specific software elements and relationships between them, to represent high-level structural and behavioral abstractions. It supports architectural patterns and anti-patterns detection. For example, it can detect and suggest resolution of cyclic dependencies between packages, or compute automatic layering of components within the system. It currently provides support for C/C++, Java, Cobol.
Contact Person: Maayan Goldstein
- Reviews and Bug Patterns
Reviews are the most effective technique known today for early detection of software problems. Reviews are known to find between 60% and 90% of system defects. They are also expected to find a major defect every 3.5 hours of inspection as opposed to every 15 to 20 hours of testing.
Review effectiveness is skill-sensitive. Diverse technical and nontechnical skills are required to conduct effective reviews. There are many review and modeling techniques, such as Fagan review, desk checking, the interleaving review technique (IRT), contract review, state machine based review, and check-list or bug patterns based review, to name a few. Familiarity with the different review techniques and the ability to apply them when appropriate is necessary for conducting effective reviews. In addition, the correct mindset and skillful leading of the review session in general and the inspection meeting in particular dramatically affect the review results. Finally, deep knowledge of the system architecture is a great plus in the review process.
The SPARQ group offers education, methodologies, and tooling for conducting effective reviews and modeling. Special emphasis is given to addressing different technical concerns, such as concurrency, interfaces and security.
Contact Person: Orna Raz
- SmartCloud Entry (SCE) Simulator
SmartCloud Entry is a cloud management tool that presents a unified interface for different clouds (VMControl, OpenStack, Vcenter). The SCE Simulator technology replaces physical clouds with simulated ones and thus, simplify development, testing, and demonstration of SCE. This technology is advantageously used by IBM SCE developers, testers and sales teams.
Contact Person: Daniel Citron
- Nova OpenStack Simulator
The Nova component of the OpenStack infrastructure handles the creation, management, and deletion of Virtual Machines. The Nova OpenStack Simulator replaces the code that communicates with hypervisors on physical computers with simulated code that mimics the behavior. This enables testing the correctness, scalability, and robustness of OpenStack without the need for physical compute nodes. This technology is being developed with the cooperation of IBM OpenStack developers.
Contact Person: Daniel Citron
- International Workshop on Combinatorial Testing (IWCT)
- MUSEPAT - International Conference on Multicore Software Engineering, Performance, and Tools
- HVC - Haifa Verification Conference