![]() |
![]() |
![]() |
![]() |
|
| Continual Optimization | |||
|
|
|||
|
|
We use the term "continual optimization" to refer to the application of advanced analytic methods, such as mathematical optimization and special purpose heuristics, on combinations of real-time and historical data, to update business plans and make decisions that are communicated in real time for the purpose of optimizing business objectives. Often these plans and decisions involve the allocation of scarce resources. We now have an unprecedented opportunity to analyze and model complex systems with increasing accuracy and precision, due to dramatic growth in the availability of processing power, advances in software architectures, the availability of robust software libraries, increases in the availability of real time digital data and fast, high-bandwidth communications capability. Most importantly, companies now need to use analytic tools as they engage in ever more competitive e-business. The rapid proliferation of e-commerce provides extensive time sensitive data (demand, supply, prices) that has not yet been subject to significant analysis or optimization. The infrastructure and technology components required to support "continual optimization" are rapidly becoming available in most industries. Companies now have an opportunity to take the lead in exploiting these capabilities and to change the business rules in their industries. To achieve the benefit of continual optimization, the analytic methods, embodied as proven, trusted software implementations must be applied to up-to-date, accurate data, with intuitive interfaces providing visualization and the ability to interact with both the input data and the solutions. However, to achieve this state, the analysis engines must be robust, and configuring business processes which use analytic engines with other tools must be trivially easy.
We expect that new business models will be enabled through analytics that allow real-time operational decision making, explicit consideration of uncertainty, and increased efficiencies attainable through aggregation and late binding of resources. To achieve this goal, we must also migrate toward robust, rather than strictly "optimal" planning, and develop models and methods for determining robust plans and, more importantly, models and methods for updating plans in response to changes in business information. In addition, significant research is required to more fully understand the advantages, and limitations, of new computing architectures, such as loosely coupled distributed clusters (grids) and massively parallel servers in the area of mathematical optimization.
Emphasis in a mathematical modeling and optimization must shift from exploring clever and easily solvable, but brittle, models and algorithms, toward the development of new models for robust optimization, including definitions of "robustness" and new methods for dealing with multiple objective functions and ranges on constraint values. As a simple example consider the problem of computing a shortest path in a graph. Computing a shortest path is quite simple, however the problem becomes more complex if we wish to compute a "robust" path such that the (possibly detoured) path will remain short even if the length of up to k edges are changed while we traverse the path. Solutions that are easy to "repair" or update as data changes along with fast algorithms for performing the updates will be required to support pervasive use of analytics for continual optimization. The design of "incremental algorithms" that make use of an initial solution in the original setting to reach a solution in the up to date setting are essential to guarantee reaction to changes in a timely manner.
The challenges to realizing continual optimization fall into several categories. The first is definition, dissemination, maintenance and synchronization of mathematical models representing the underlying problems to be solved in the pervasive models of continual optimization. The second is managing the dataflows generated by the models as they collect and incorporate real time data into the modeling framework. The third is developing algorithms for maintaining a model state representing a high quality solution while cognizant of the constraints placed by data transfer limitations and the dynamic nature of the models themselves.
Definition, dissemination, maintenance and synchronization of mathematical models will begin with the definition of standard representations for the static and structural parts of models and their interfaces to the collection of models into which they are delivering information, as well as the message and event frameworks into which they must fit. Due to the asynchronous nature of continual optimization, models will act as agents collecting information processing it and pushing it on to their subscribers. Existing standards (e.g., MPS file format) and commercial software tools should provide a starting point for the necessary standardization, but a significant amount of additional work is required, including extensions required to deal with ranges of data, compact descriptions of alternative solutions, and stochastic data. Additional standards are also required to specify how different tools (both current and to be developed) interact with one another.
Managing the dataflows generated by the analytic engines and the applications that use them will be critical to the viability of any large continual optimization system as growth in the number of participants may produce tremendous growth in the volume of information resulting in system instability. This could involve exploitation of the property of many algorithms that allows the checking of a solution to be done (possibly) much faster and with much less data than the computation of a new solution. Additional approaches that exploit mathematical properties of models and algorithms should also be developed.
Most importantly, we must also develop hierarchical, incremental, and robust algorithms. Continual optimization in many cases requires modeling of complex decision systems that are hierarchical in nature. This will make us revisit classical decomposition algorithms as well as develop new algorithms for handling decomposition with asynchronous messaging and dynamic collections of subproblems. Algorithms which can capitalize on the existence of a solution (or near solution) and improve that solution using fast heuristics will dominate parts of the continual optimization model. The requirement for stability of the solution process as well as implementability of solutions will require development of approaches which produce good robust, but not necessarily optimal, solutions. Indeed, with the dynamic nature of continual optimization, the lifetime of a solution may be very short especially if it is fragile. |
| About IBM | Privacy | Legal | Contact |