IBM®
Skip to main content
    Country/region [change]    Terms of use
 
 
 
    Home    Products    Services & solutions    Support & downloads    My account    
IBM Research

Computer Science

Innovation Matters


Performance Modeling and Analysis

AMBIENCE - Automatic Model Building using InferENCE

Selected Publications

A smart hill-climbing algorithm for application server configuration, Bowei Xi, Zhen Liu, Mukund Raghavachari, Cathy H. Xia and Li Zhang, WWW 2004.

Abstract:
The overwhelming success of the Web as a mechanism for facilitating information retrieval and for conducting business transactions has ledto an increase in the deployment of complex enterprise applications. These applications typically run on Web Application Servers, which assume the burden of managing many tasks, such as concurrency, memory management, database access, etc., required by these applications. The performance of an Application Server depends heavily on appropriate configuration. Configuration is a difficult and error-prone task dueto the large number of configuration parameters and complex interactions between them. We formulate the problem of finding an optimal configuration for a given application as a black-box optimization problem. We propose a smart hill-climbing algorithm using ideas of importance sampling and Latin Hypercube Sampling (LHS). The algorithm is efficient in both searching and random sampling. It consists of estimating a local function, and then, hill-climbing in the steepest descent direction. The algorithm also learns from past searches and restarts in a smart and selective fashion using the idea of importance sampling. We have carried out extensive experiments with an on-line brokerage application running in a WebSphere environment. Empirical results demonstrate that our algorithm is more efficient than and superior to traditional heuristic methods.


Analysis of Performance Impact of Drill-down Techniques for Web Traffic Models, Cathy H. Xia, Zhen Liu, Mark S. Squillante, Li Zhang, and Naceur Malouch, Proceedings of the 18th International Teletaffic Congress (ITC18), Berlin, Germany 2003.

Abstract:
The performance of Web sites continues to be an important research topic. Such studies are invariably based on the access logs from the servers comprising the Web site. A problem with existing access logs is the coarse granularity of the timestamps, e.g., arrival times. In this study we demonstrate and quantify the significant differences in performance obtained under diverse assumptions about the arrival process of user requests derived from the access logs, where the corresponding user response times can differ by more than an order of magnitude. This motivates the need for a general methodology to construct accurate representations of the actual arrival process of user requests from existing coarse-grained access-log data. Our analysis of the access logs from representative commercial Web sites illustrates self-similar behavior of the arrival process, and we exploit the properties of these self-similar processes as a theoretical foundation for constructing the arrival process at finer time scales. The advantage of our approach is that it maintains consistency between the properties of the arrival processes at both coarser and finer time scales. In addition, our analysis of the request size distribution from commercial Web sites demonstrates a sub-exponential, but not heavy-tail (power-law) distribution. Through simulations, we investigate the impact of these different traffic models on the user request response times.


Parameter Inference of Queueing Models for IT Systems using End-to-End Measurements, Zhen Liu, Laura Wynter, Cathy H. Xia and Fan Zhang, Performance Evaluation, to be appeared.

Abstract:
Performance modeling has become increasingly important in the design, engineering and optimization of Information Technology (IT) infrastructures and applications. However, modeling work itself is time consuming and requires a good knowledge not only of the system, but also of modeling techniques. One of the biggest challenges in modeling complex IT systems consists in the calibration of model parameters, such as the service requirements of various job classes. We present an approach for solving this problem in the queueing network framework using inference techniques. This is done through a mathematical programming formulation, for which we propose an efficient and robust solution method. The necessary input data are end-to-end measurements that are usually easy to obtain. The robustness of our method means that the inferred model performs well in the presence of noisy data and further, is able to detect and remove outlying data sets. We present numerical experiments using data from real IT practice to demonstrate the promise of our framework and algorithm.


Profile-based Traffic Characterization of Commercial Web Sites, Zhen Liu, Mark S. Squillante, Cathy H. Xia, Shun-Zheng Yu, Li Zhang, Proceedings of the 18th International Teletaffic Congress (ITC18), Berlin, Germany 2003.

Abstract:
The problems of workload characterization, performance modeling, workload and performance forecasting, and capacity planning are fundamental to the growth of Web services and applications. Previous studies have primarily focused on the complexity of Web traffic at the level of low-order characteristics such as object-hits or page-views. In contrast, our study focuses on higher-order characteristics, and introduces techniques for profiling, clustering and classifying commercial Web sites based on these higher-order traffic characteristics. Specifically, we devise a methodology for the extraction of Web traffic patterns from access logs, the clustering of such traffic patterns, and the classification of Web traffic based on this extraction and clustering of traffic templates. Our methodology has been success-fully applied to accurately capture and characterize the complexities of traffic exhibited at more than 25 production commercial Web sites. These methods provide new solutions that can then be exploited to address challenging problems such as workload and performance prediction, and short-term and long-term capacity planning.


Web Workload Service Requirement Analysis: A Queueing Network Approach, Li. Zhang, Cathy H. Xia, Mark S. Squillante, W. Nat Mills, MASCOTS 2002.

Abstract:
The answers to many important performance relatedquestions with multiclass queueing models depends uponhaving estimates for the service times of different classesof jobs. We present a general approach to infer the per-classservice times at different servers in an environmentwhere only server throughput, utilization and per-class responsetime measurements are available. The per-classservice times are solutions to an optimization problemwith queueing-theoretic formulas in the objective and constraints.We further study the impact of the variance of servicetimes on the variance of response times. A few casestudies are presented to demonstrate the power of our approach.




    About IBMPrivacyContact