Impact of service-oriented architecture on enterprise systems, organizational structures, and individuals

Initiatives for service-oriented architecture (SOA) and on demand business are being adopted at various corporations to meet the operating challenges of business in the 21st century.^1–3 Currently, the primary focus is to apply SOA concepts incrementally to existing information technology (IT) systems to exploit short-term business benefits. To unleash SOA's full potential, however, a broader vision and perspective is required—one that permeates and transforms the fundamental layers of an organization.

SOA facilitates aligning existing IT infrastructure and systems to achieve end-to-end enterprise connectivity by removing redundancies, generating unified collaboration tools, and streamlining IT processes. In addition, institutionalizing both operational and technical governance and equipping them with adequate funding commitment are crucial success factors. To effectively leverage the virtualized IT services layer and its collaborative tools, the organizational model must be transformed to create differentiated and flexible team-based services. The new model optimizes cross-business unit operations to deliver objectives, eliminates costly duplication, and flattens management chains. The resulting structure is flexible, agile, and well-orchestrated.

Incorporating organizational redesign is a very complex task. It involves executing a well-planned transition strategy that harmonizes existing elements in the new structure. This is done by modifying existing traditions, by instituting new human resource (HR) practices and management principles, and by developing novel incentive arrangements. All of these changes need to be performed without major disruptions to the existing corporation fabric.

Corporate initiatives and directives are often necessary to induce the required behaviors in a company to successfully support an enterprise-wide SOA. These initiatives include establishing IT directives for creating business transformation, creating executive councils and architecture boards, institutionalizing governance policies and models, and most importantly, allocating funds to sponsor these directives.

IT transformation initiatives are needed to create an on demand business and an IT environment that is nimble, robust, and less expensive. In this paper, we present several initiatives that provide insight into the enterprise transformation undertaken by IBM's on demand strategy. One such initiative, ECBA (Enterprise Component Business Architecture),⁴ represents the key framework for IBM's SOA adoption.

Given the realities of running a corporation, successful SOA implementations are affected by direct or indirect funding controls that enable business-unit activities to focus on projects that promote and sustain a coordinated and coherent company-wide architecture.

After the business value benefits for SOA have been calculated for return on investment (ROI) and IT efficiencies (e.g., reuse and reduced development costs), a corporation's economic behaviors have to be adapted to support the SOA vision. New initiatives need to be institutionalized to guide the organic evolution of the enterprise toward this vision. Because many large enterprises operate as “silos” of individual sub-brands, a corporation's economic directives must promote a collaborative environment of IT goals that support SOA-driven, enterprise-wide IT capabilities.

Funding models are highly dependent on the financial resources of the owning business unit and pose obstacles to efficient funding decisions for enterprise-wide services. Should the business units primarily control funds for IT services, should they be centrally controlled at an enterprise level in the Chief Information Officer's (CIO) office, or should there be a combination of these approaches?

The company's financial planning activities to create IT functions and services must be coordinated at an enterprise level by the CIO to produce and maintain a set of common services that can be used by more than one business unit. To maximize the efficiency of IT implementations, a balance must be established between business-unit autonomy and enterprise-level directives for coordinated IT investment strategies. Organizational redesign may be required to streamline IT investment and institute a system of councils to define and manage IT services.

The CIO's office has the responsibility to define the strategic enterprise architecture that provides the infrastructure for IT activities and architectures in each of the company's business units and to illustrate how this on demand environment enhances the business unit's capabilities and competitive advantage. A collaborative relationship must be explicitly defined between the CIO's office and the business units, and among the business units, to ensure an integrated end-to-end approach in creating a services-based infrastructure.

The IT activities of the business units must be coordinated at an enterprise-wide level to maximize service reuse and eliminate redundant implementations. The reuse of common IT services (both inter- and intra-business units) is a critical success factor for an SOA. Reuse promotes company-wide consistency of key business operations and processes, while reducing costs. It is indirectly impacted by cultural proclivities (to reuse rather than create) in the technical community and directly affected by cross-business-unit cooperation and collaboration.

Common enterprise services must have defined owners with established ownership and governance responsibilities. These owners are responsible for gathering requirements, development, deployment, the boarding process, and operations management for a service. The service must meet the functional objectives within the context of the business unit and the enterprise. Each service ownership is associated with a business scope. Examples of such business scopes are customer relationship management (CRM), customer information and entitlements, order management, financing, and taxes. Intra-enterprise services, such as employee portals, collaboration tools, procurement and expense tools, information repositories, and intellectual capital repositories, are other essential functions that also need identified business owners. It is critical to specify an executive as the owner for each logically connected set of services. The owner's responsibility is aligned with the overall enterprise governance.

Enterprise governance can take many forms. Peter Weill and Jeanne W. Ross, of the Massachusetts Institute of Technology Sloan School of Management, have defined IT governance as “specifying the decision rights and accountability framework to encourage desirable behavior in using IT.”⁵ Internal IT governance is often a sensitive topic because it is usually perceived by the technical community as introducing more complexity and impediments than benefits. For efficient SOA deployments, it is critical to streamline SOA-related project controls to the bare essentials and promote service reuse, technical consistency, and interoperability. It is crucial to maintain a centralized control of common business object schemas and semantics definitions and to enforce consistent change-management policies.

Often, the most visible forms of project governance are internal technical standards to which projects must adhere during the project life-cycle phases (inception, elaboration, construction, and deployment). The technical standards could be modeling and architectural methodologies, best practices and patterns, or technology adoption guidelines (such as Web Services). Nevertheless, unless the benefits of these standards are well-communicated and enforced, teams may find ways to circumvent what is perceived as unnecessary. Peter Weill gives broad-range examples and guidance from extensive enterprise analysis.⁵

The office of the IBM CIO established an Enterprise Architecture Council, which manages IT governance. This encompasses process design management, data architecture management, application architecture management, and infrastructure. These areas must be coordinated and integrated to ensure that there is collaboration in the efforts to identify and reuse services through the project life cycle.

A unified view of a company's architectures is a critical foundation upon which to define a coordinated direction for integration of IT services. A company-wide architectural blueprint can illustrate the use of enterprise and cross-business-unit components in various detailed architecture views (along with service definitions and interface protocols).

The effect of these enterprise- and business-unit-level architectures on IT projects must be managed by documented processes. Enterprise-level architectures provide coordination and a unified view across multiple business-unit-level architectures. Some project-level governance can be provided by a business transformation management system that includes checkpoints to determine compliance with technical standards and policies.

IBM's Enterprise IT Standards and Corporate Instructions primarily dictate controls for security, privacy, business interactions, technology usage, project management, processes, data, and applications. Typically, these standards and instructions are manually created and instituted after evaluation and assessment. Interface standards need to be established to allow for effective decoupling to enable business partner integration and potential outsourcing opportunities. This includes messaging standards and object schemas that define interface semantics which, in turn, define a consistent and reusable set of request and response artifacts. Internal SOA-related standards should be based upon industry standards and specifications to allow for service interoperability with external business partners.

Enterprise-level technology guidance is often needed to provide a consistent and interoperable system of available services in addition to maximizing development efficiency.⁶

Internal technology standards are useful in providing templates for projects to help create standardized, readily accessible services that are easily consumable by other clients or service applications. It is helpful for the standards to define the conditions under which specific programming models or protocols must be used. These standards not only dictate technology and product choices, but also methodologies and deployment platforms. Although not immediately apparent, this standardization has advantages that can more than compensate for the reduced level of design flexibility for development teams. A common set of interface protocols and development tools can minimize the level of training and maximize the pool of available developer resources to create and use services. Accompanying cultural and behavioral transformations are often necessary to reinforce this.

As a standard design practice, ECBA involves first designing process models (using WebSphere* Business Integration Process Modeler) that have built-in services references by use of reusable process model segments called process modules. These process modules are then translated into a direct input for workflow modeling tools (WebSphere Studio Application Developer—Integration Edition), which generate runtime workflows and artifacts. The workflows then access the services of ECBA's distributed components, which are required to have service-based interfaces. The distributed components are compatible with several established definitions for components. (See Reference 7 and Reference 8.)

IT initiatives can derive a considerable amount of value from pattern-based approaches. In addition to well-recognized design and architecture patterns, emerging e-business patterns can accelerate the development of initiatives by taking advantage of recently established models of business interactions in an electronic world. (See Reference 9 for an introduction to the concepts of business and integration patterns.) These concepts enable a simple, yet powerful means to systematically organize and coordinate the identification of required IT functions and their interactions. The business patterns are self service (user-to-business), information aggregation (user-to-data), collaboration (user-to-user), and extended enterprise (business-to-business). They are coordinated with two integration patterns: access integration and application integration.

Grass-root technical efforts involving community portals and developer interest groups within a company allow IT practitioners to share learning experiences and intellectual capital artifacts. This is instrumental in disseminating knowledge about a company's SOA skills, tools, collateral, and products. The technical portals should also share the strategic technology directions and standards and be continually updated to ensure relevance. These community interactions are not a replacement for formal education programs. Education programs and classroom sessions are critical to disseminating emerging concepts and bridging skill gaps.

Official technology boards can provide vehicles to obtain consensus for standards and technology choices. Examples of such IBM enterprise-wide groups are councils for Web Services and XML (Extensible Markup Language), composed of key IBM architects from each business unit.

IT governance standards and instructions can be represented in declarative format, such as XML, which then can be used by the design models and the generated runtime artifacts to enforce compliance. Alternatively, services (which are accessible during runtime) can be created to verify adherence to these corporate standards. For example, there is an ECBA enterprise service which validates government export compliance and checks the U.S. Bureau of Industry and Security Denied Party list.

A level of risk is associated with adopting leading-edge technologies in SOA implementation. Progressive projects or business units must balance the benefits and potential costs associated with emerging technologies in new product releases. This higher cost can be associated with increased interaction with product support teams during the project life cycle. Company infrastructures may not be prepared for new technologies or product versions, which may result in charges that are temporarily higher than normal for out-of-profile, sole product deployments that are not part of the standard infrastructure package (whether hosted internally or by an external provider).

Internal standards need to be updated constantly to reflect emerging specifications and technologies (especially in the fast-paced Web Services arena).¹⁰ Legacy standards need to be modified to remove barriers to successful SOA development and deployment. Adjustments are required to accommodate emerging infrastructure concepts, such as the Enterprise Service Bus (ESB)¹¹ and the IBM On Demand Operating Environment.¹²

A flexible, virtualized IT infrastructure is required to rapidly respond to on demand needs. A collaboration of several IBM business units has defined the On Demand Operating Environment (ODOE) to provide a set of integration and infrastructure-management capabilities to enable this rapid response. These modularized capabilities can be selected as needed and combined into various solutions to satisfy the needs of a company's on demand business initiatives. The ODOE (see Figure 1) enables a multidimensional infrastructure framework to facilitate SOA, support pluggable application services and business processes, and create business partner services, choreographed processes, and utility and resource virtualization.

Figure 1

The need for an SOA-capable infrastructure that supports the unique requirements of service operational management is often overlooked until after an initial set of services has been deployed. Experience has shown that traditional IT management products cannot support maintenance of service-based systems. Investment in an ODOE is critical for implementing service access management products in conjunction with other operational tools that monitor and manage service responses and requests. Enterprise repositories, such as Universal Description, Discovery, and Integration (UDDI), and approaches based on the Reusable Asset Specification (RAS), provide support for an enterprise-wide, systematic, and regulated pattern of reuse.

An integral part of the ODOE is the Enterprise Service Bus (ESB), which allows for the virtualization of services though a middleware intermediary that performs transportation services (e.g., for secured and assured delivery), mediation services (e.g., routing, protocol, and data transformations), and event management. It is essentially a middle layer that resides between service requestors and service providers and adds value by relieving the requestors and providers of various functions provided by these ESB services.¹³

Collaborative tools are essential to maximize the efficiency of interactions among multiple groups and individuals. The IBM on demand workplace (ODW)¹⁴ provides a work environment that seamlessly incorporates collaboration activities into a person's daily activities and processes. Employees are able to communicate and collaborate faster with other employees, customers, and business partners by using dynamic, role-based interfaces to access critical work tools. The ODW enables solutions for a collaborative environment that includes employee portals, e-learning, employee self-service, and expert directories and their associated content management.

The ODW uses an integrated collection of IBM software and services to improve employee productivity and reduce costs. It includes extensive collaboration functions to allow individuals to participate in operational business processes. Disparate portal efforts can be combined into a single, consistent, and easily managed solution that uses WebSphere Portal to simplify access to content, applications, business processes, and people.¹⁵

A strong advantage lies in ODW's ability to leverage existing IT resources and to incrementally deploy capabilities as they are required by business needs. SOA influences are evident in the IBM Workplace Collaboration Services,¹⁵ which offer a set of prebuilt, reusable collaborative services that simplifies how information is gathered and shared. Other technologies, such as team rooms, resource and skill management tools, and employee directories can also be consolidated into an enterprise-wide workplace to increase the effectiveness of the workforce.

Organizational structures are usually modified and adapted to handle significant changes in the operating business environment in order to maintain a competitive advantage. SOA and ODOE have provided a revolutionary IT framework to fulfill the goals of many CIOs to achieve end-to-end enterprise and interenterprise connectivity by providing a transformed technology infrastructure that efficiently meets the needs of internal and external IT services.

To maximize the benefits of this IT transformation effectively, an organization structure that is perfectly aligned to exploit this new IT service framework is required. This new structure has to be optimized to meet business agility needs, streamline tasks and associated communication, minimize management overhead, and provide result-oriented outputs, and all of this must be done in a truly global and dispersed structure and employee base.

There are several formal organizational structures in practice today.¹⁶ These structures are adopted based on the unique requirements of the corporation; each has its merits and disadvantages. The following subsections describe some of these structures.

This structure is usually found in a company that is small and handles a single function or product line. The customer needs are uniform, and the activities are simple, repeatable tasks. The functional hierarchy benefits from centralized control, promotes niche expertise, and enhances operating efficiency.

Such structures are not efficient, however, for larger enterprises because centralized control introduces multilayered bureaucracies, impedes agile decision making, and fosters disruptive interfunctional rivalry. The communication channels are hierarchical and encourage myopic management. Functional hierarchies are usually not able to respond to rapid changes in business conditions.

Large enterprises typically adopt this structure to address the unique demands of a geographical market or to capitalize on the economies of local operations. It projects a comfortable image of proximity and provides collocation for efficient delivery and customer support. On an international scale, this structure can be customized to accommodate government regulations and local customs.

The geographic diversity of this structure introduces problems such as incompatible strategy execution, duplication and cost disadvantages, and inconsistent marketing messages. Frequently, area managers press for more freedom to define strategies, and another management layer is required to contain this pressure and maintain uniformity.

There are two variations of this structure: the decentralized line of business and the strategic business unit. The first is centered on products and services, and the second is more focused on executing strategic imperatives. The divisional structure provides a mechanism to decentralize responsibility and delegate authority and ownership. There is clear accountability, and there are well-defined performance metrics. The cohesiveness within the division allows for efficient coordination of tasks and activities.

This divisional autonomy runs the risk of encouraging the silo effect, resulting in negligible coordination of related activities and costly redundancies. The senior corporate executives, heavily dependent on the divisional heads, become unfamiliar with the tactical execution. Sometimes the divisional units are not well-defined or grouped for administrative convenience.

A matrix organization has been embraced by large enterprises that have a diversified product and services portfolio and, typically, a global presence. This structure provides a framework for checks and balances and formal handling of strategic priorities. It encourages cooperation, consensus building, and coordination of related goals and activities among cross-functional units.

Effectively managing a matrix organization is very complex. It is difficult to maintain balance among the subsets of the organization. An inordinate amount of time is typically spent on communication channels, which are usually clogged with nonessential messages. The shared authority usually results in little or no progress because synchronizing clearances required from each authority impedes decision making, thus reducing business agility.

In the fast-paced constantly changing on demand environment, businesses are struggling to reduce market cycles, enhance customer satisfaction, increase revenue, seize competitive advantage, respond effectively to emerging threats, and adapt swiftly to swings in the business atmosphere, at the same time trying to reduce operating costs and overhead.

The ODW¹⁴ provides employees with the necessary tools for effective job execution and improves productivity, in turn reducing costs. Nevertheless, for a global enterprise, the limitations of the current organizational structures prevent it from being aligned completely to exploit and fully realize the IT transformation benefits.

A parallel transmutation similar to the transformation of IT systems by SOA and on demand initiatives is required for organizational structures.^17,18 IT systems have evolved from mere tools and accelerators to an organic organizational entity. This new entity needs to be factored into the proposed structural design. The 21st-century business challenges described by Drucker² can be addressed by applying the SOA metaphor to organizational redesign.

The essential characteristics of an SOA-based service are its levels of abstraction, clear, fully described contractual interface, and easy discovery and invocation. The services can be further composed and choreographed to assemble more complex services with similar key characteristics. These services are implemented with focal emphasis on satisfying the contractual interface, managed and executed in a scalable and resilient IT environment, and operated by adhering to governing policies and service-level agreements (SLAs).

Applying the SOA metaphor to organizational structure intuitively leads to viewing core tasks and activities as units of service. Each team provides a service and is specialized in delivering a particular activity or task. A chain of services from various teams can be orchestrated to execute higher-level tasks or business objectives. The services teams and their core competency are publicized on an internal bulletin board housed within the ODW. Also published are the governing guidelines and policies which specify how to engage their services.

In a large corporation, to identify such differentiated services might necessitate defining numerous teams and corresponding niche services, and these teams would need to be managed with an effective orchestration and coordination facility. These virtualized services would need a collaboration and coordination fabric, and this is provided by a logical ESB through the facilities of the ODW tools coupled with well-defined service engagement and communication channels. We call this coordination facility the collaboration-and-orchestration bus (COB). In this paper, a bus is a conceptual artifact that is used to connect interrelated entities by providing communication, coordination, and collaboration mechanisms. The team services, the composed team services, and the collaboration-and-orchestration bus comprise the Human Services Bus.

The Human Services Bus (HSB) is an optimized organizational service structure designed to meet the needs of the on demand business environment. See Figure 2. It derives its structure by extending the traditional SOA semantics and including the existing organizational structures, merging them to maximize advantages and reduce limitations.

Figure 2

The central logical entity in the HSB is the service. A service can be anything that executes a particular task that delivers objectives, tactical results, or strategy realization. Services can be further aggregated to compose larger, complex services. Figure 3 depicts the various service layers within an HSB and their functions.

Figure 3

Additionally, for the services to perform optimally and to streamline their operation, service agents need to be defined. These are individuals who are identified to monitor, mediate, or choreograph services. Their roles and responsibilities vary depending on the layer and are critical to the HSB. In the following subsections, we describe in detail the groups of services shown in Figure 3, starting at the bottom of the figure.

These are the most fundamental services within the HSB. These services are clearly defined to deliver tasks and perform activities relating to the organization's core competencies. The tasks of the team services group are narrow and precise, such as “functional testing of component A in product XYZ,” “data access performance benchmarking in the retail industry,” and “level 1 customer support for component B in product XYZ.” The service agent for this layer is a manager whose responsibility is to mediate and ensure that the service is operational and meeting contractual requirements, to optimize linkages to the collaboration engine, to troubleshoot day-to-day issues, and to regularly handle team incentives and morale.

Team services are aggregated at the departmental level to deliver core business objectives and thus create departmental services. Examples of these services are “testing component A in product XYZ,” “performance benchmarking in the retail industry,” and “customer support for component B in product XYZ.” Senior managers are the first-level service choreographers; that is, they are responsible for understanding the business objectives delegated to them, creating a service workflow based on the existing services to satisfy those objectives, and ensuring that the workflow connections are streamlined by liaising with team service managers.

These services are created by choreographing the departmental services to execute the tactical goals of the company. Examples are “testing of product XYZ,” “industry performance benchmarking,” and “customer support of product XYZ.” The business-unit services can also meet some of their key requirements by leveraging the team services directly, thus supplementing the orchestration of the departmental services. Crafting business-unit services that execute and deliver the tactical elements and manage the key profit-and-loss results requires executive skills. A director is responsible for choreographing the departmental- and team-level services to meet these needs. He or she teams with the service agents in the layers below to evaluate the services which his or her unit uses, locates process bottlenecks, reengineers the service characteristics when needed, eliminates service redundancies, and defines new emerging services.

Business-unit services need to be coordinated to drive the organization's strategic objectives. The divisional services may include “manage product XYZ,” “manage industry solutions,” and “customer relations.” The designers of divisional services have the gamut of team, departmental, and divisional services at their disposal and can leverage them at their optimal granularity. Senior-level executives, such as vice presidents and general managers, are responsible for realizing strategic objectives by orchestrating divisional services and optimally channeling funds.

These services are focused on executing the overall mission and goals of the organization. They define the strategies and manage them by orchestrating the divisional services. Examples could be “software portfolio services” or “industry-solution services.” Senior vice presidents work with the CEO and his corporate team to define the periodic goals and set the overall organizational strategy. Each senior vice president monitors the efficiency and results of his or her divisional and business-unit services and creates directives to achieve those goals.

The size of the teams handling the services depends on the activity, scope, and sizing of the service. Certain team services that have a high demand (e.g., “administrative and secretarial services”) can be replicated for support by multiple service teams. The demand for a particular service is driven by the on demand needs of the business and the costs.

Defining services with differentiated activities and distinct results helps streamline a large organization, remove redundant teams, and reduce expenses. It also facilitates service discrimination by helping to eliminate less strategic services and create new services. This makes the organization very agile, adapting quickly to new opportunities and competitive threats.

Both IT-centric and human-centric services are externalized in the same manner, and the service contract defined by the interface abstracts the main features of the service. Both of these kinds of service can be seamlessly choreographed to compose more complex services.

The HSB also contains a communication and collaboration framework, which is the IT engine necessary to support the HSB logical structure effectively. The COB is analogous to the ESB in an SOA-based IT application. The COB is derived by customizing the ODW tools and ODOE.

The COB provides the IT infrastructure to formally advertise team services and offers workflow tools to support joint activities and coordination across services (and teams) and monitor task completion and early crisis detection. The service agents at each layer are supplied specific planning and design tools to discover, orchestrate, and choreograph niche services from the lower service layers. Executive “dashboards” and configurable reporting tools provide snapshots of various service flow metrics and productivity data.

The services are virtualized, and the individuals on the teams supporting a service may be geographically dispersed. To support teaming, a portfolio of collaboration tools is also provided by the COB. A comprehensive array of synchronous and asynchronous tools forms the COB messaging backplane. Facilities such as e-meetings with electronic whiteboards, instant messaging, webcasts, and task-oriented community tools supplement the existing synchronous communication facilities, such as teleconferences. Asynchronous communication is supported by specialized team rooms, project databases, interactive team portals and forums, and e-mail.

Discovering a service, being able to understand its capabilities, and leveraging it optimally are crucial. The discovery, evaluation, and orchestration tools used by the service agents are provided by the COB.

Currently, an organization's capabilities are often inadequately shared, using a mix of ad hoc measures—passive Web sites, private bookmarks, or, in some cases, personal notes.¹⁹ In a large organization this leads to replicated efforts and overlapping tasks. Sometimes these inefficiencies are hidden in deep meshes and shrouded by middle management layers. The tools described in the following subsections replace this chaos with an orderly, efficient utilization of capabilities.

Service directory tool
The service directory tool (SDT) is a standards-based tool with which all services (including aggregated services) in an enterprise are described normatively and published. The services are also annotated exhaustively with key characteristics, such as service delivery guarantees, sample outputs or references, current stakeholders and team members, and ratings. The rating is a critical metric in aiding a potential stakeholder (such as an executive) to discriminate among services and make orchestration decisions.

The rating meta-data is defined when a service is created. Based on the service type, subrating meta-data would be defined for both functional characteristics (e.g., for a component testing service, such characteristics as test report quality, number of defects identified, or test intervals) and nonfunctional characteristics (e.g., turnaround time or reliability). The overall service rating is a weighted average of the subrating values (these are derived at service creation time and fine-tuned regularly). Using the SDT's autonomic facilities, users and stakeholders of the service provide feedback and populate the rating values on a transaction basis. When higher-level services such as departmental services are invoked, the ratings are propagated to the respective team services. Additionally, the rating information can be cascaded upward to gather the key metrics to evaluate departmental or business-unit performance and efficiency, providing senior executives with the “pulse” of the organization.

The SDT supports browsing and allows all individuals to obtain access to service information and to expand and drill down from higher-level services. The access rights are governed by their roles, authorization levels, and privacy policies.

Asset directory tool
Certain services, when invoked, produce work products and assets (e.g., product binaries, architectural blueprints, best practices, and technical documents) as responses. Sometimes the work products are linked to the service choreography and are transmuted at each step into polished and reusable assets.

These work products and reusable assets are housed and publicized by the asset directory tool (ADT). Asset-producing service teams are given reusability directives by their service agents. When repeatable requests are placed by other services, they can be responded to by ADT links. Also, stakeholder service teams are directed to search the ADT for relevant assets before invoking asset-producing services to minimize unnecessary communication. This asset-request match feature can be supported by the COB.

The ADT hosts both confidential and nonconfidential intellectual capital. Nonconfidential and shareable assets can be promoted on extranet and customer-relationship portals. Access rights and controls are used to protect confidential assets.

Employee directory tool
The employee directory tool (EDT) is essential in an on demand environment. Because teams are dynamically assembled and orchestrated to execute business objectives, individuals are provided only a short time to acclimate and adjust to the team atmosphere. All employees are listed on the EDT.

The EDT is a focused tool for employees to share personal and professional information: experience and qualifications, prior and current roles and responsibilities, comprehensive contact information, personal calendar and schedules, reporting structures, accomplishments, linked contact information (direct formal and informal relationships with other employees), photographs, and personal “blogs” (Web log consisting of frequently updated chronological entries on a particular topic). Although most of the information is generally accessible, an employee may choose to constrain access rights to private and confidential information.

Putting a human face on team members and creating virtual collocation for a distributed team is the key intent of the EDT. This and the ODW collaboration tools equip employees to create formal and informal networks that are essential for effective teaming and individual growth. The SDT, ADT, and EDT are cross-referenced to facilitate information traversal and have exhaustive search indexes.

HSB adoption provides the following benefits:

1. Optimized business operations at reduced cost
2. Alignment with the customer and enhanced business agility
3. Streamlined internal business operations
4. Individual roles and growth opportunities

The focused approach in modeling the organization along a service-based structure enables quick identification of defunct services, enhancement of vital services, and discovery of core service gaps. The service agents are directed by concrete responsibilities within each layer. Automated productivity monitoring (through the rating system) ensures proactive mediation. In addition, the traditional middle-management “spaghetti” (complex and tangled control structure) of older structures is unraveled, and the redundancies are removed.

Senior executives can launch their COB dashboards to view their service structure's operational health, monitor progress, and refine orchestrations to resolve issues. The dashboards provide snapshots of both real-time and historic data, which help the executive in operational analysis.

These capabilities result in optimized cross-business-unit services aligned with strategic and tactical business operations, leading to substantial cost savings from a lean organizational footprint.

Meeting customer and partner demands with proactive and efficient services helps increase customer satisfaction and loyalty. Customers are provided a set of collaborative tools and relationship portals for interacting with core partner services and become stakeholders for the externalized services offered.

Service requests enter the COB and are handled by the relevant internal services. The requests are efficiently routed to the target service and processed through the service choreography, and the output is made available according to the SLA policy.

Because services are easily configured by customer relationship managers (usually senior managers or business-unit executives), they can be adapted immediately to meet customer demands. Services can be customized to provide special attention to preferred business partners and alliances and activities, such as joint market launches, analyst briefings, technical collaboration, or premium customer support. Furthermore, partners can provide their own services in the COB pluggable framework to create value-added interenterprise process chains. These partner services can also be used to generate income by applying revenue models to them.

Competitive threat information gathered from various market intelligence services is proactively fed into the COB, and the relevant business unit or divisional services can assess the extent of the threat. Eventually, measures to thwart such threats can be instrumented quickly by either reengineering the necessary service orchestrations or by creating new services at the relevant levels.

All HSB services are geared to support the organization's tactical and strategic business operations. Currently, when new significant strategies are devised, they are usually deferred because of organizational challenges required for their implementation. The HSB provides a palette of services representing the company's core competencies, which can be orchestrated in innumerable combinations. New core competencies can be identified to supplement the existing pool.

Because the effort required for process realignment is minimal, executives are encouraged to undertake innovative endeavors. The just-in-time feedback provided by the COB autonomic instrumentation (such as the rating data) provides insights into the results and leads to low opportunity costs.

The services structure virtualizes human resources and encapsulates the individual. Because the service is measured by how it behaves and performs, this abstraction helps the employee to exercise greater creative freedom and channel his or her skills in an innovative fashion to help meet service contract requirements.

The services and individual relationships can be of one-to-many, many-to-many, or many-to-one cardinality. An employee may take any of the following paths for career growth: obtain deeper specialization in an area, take broader responsibility in serving the area services, establish higher technical skills, or sample a broader range of skills by participating in various services (e.g., within a particular orchestration at the business unit level) and acquiring orchestration skills.

The collaborative tools foster a broader participation by the service teams, unfettered by tight management controls, allowing individuals to exploit their core talents, skills, knowledge, and experience. These tools promote transparency in both service-team interactions and executive decision making, increasing the coordination of each individual's efforts with the company's business operations. Individuals are encouraged to propose and implement better methods to execute team services.

This collaboration is aligned with the ideas of Charles Handy.²⁰ Handy's proposal advanced the concept of supporting business operations with individual talent and creativity rather than by using prescribed management techniques. Teams can be comprised of diverse personality types and temperaments, as classified by Gunter Duck²¹ and David Keirsey,²² but are collectively focused on meeting service requirements and enriching the service.

Organizational changes impact how the individuals in an organization do their work and how they relate to one another. The HSB provides a whole new model for the ways in which individuals relate to services and vice versa—the individuals themselves are considered reusable assets on the HSB.²³ Most change efforts fail or are stalled because they do not address individual behavior and the collective culture changes required by business transformation of significant magnitude.

Business transformation should also address sociological and psychological perspectives. Before dealing with the specific issues related to the transition to an on demand operation, we begin by describing recent approaches to change management. We also present an example detailing the behavioral and cultural changes implemented to support the ECBA undertaking.

Eisenstat et al.²⁴ proposed an “opportunity-based design” to smooth the negative effects of organizational change. According to their view, the opportunity that is selected for implementation must be one that clearly cannot be mastered in traditional ways, thus necessitating change. Clearly, a corporation's SOA transformation falls in this category. The relationship between an individual and his or her service agent (manager) will be entirely different in the HSB environment. The individual will operate more autonomously and tend toward self-driven work. The source for performance feedback will be the COB infrastructure, and this will be monitored collectively by the employee and the manager.

Change projects often fail at stable organizations with many long-standing practices. These practices include certain forms of behavior and expectations.²⁵ Lawson et al.²⁶ advise in such situations to allow employees the necessary time to accept and practice the change. This should be complemented with consistent role models and reinforcing systems, which are catalysts to cascading change. Most important perhaps, Lawson found that “transpersonal psychology” can speed up cultural change and make it more enduring.

“Cutting the wires” is the motto with which John Brown et al.²⁷ introduce the new management styles required in knowledge worker organizations. For example, a manager's controlling task turns into a service agent's orchestrating role. Instead of exerting control that involves detailed approval for every move and decision, the new culture requires close monitoring only for awkward deviations or substantial business conduct violations. The business conduct guidelines set rules and policies for general conduct, targeting macro behavior but not micro behaviors. To enable and accelerate the HSB transition, these changes must be detailed and communicated to the corporation.

Human resource considerations that arise when introducing SOA-based IT infrastructure and governance and the new organization include the following:

1. Fostering teamwork—Interpersonal contacts are affected by any organizational change, for example, an individual's formal, informal, and trust networks. These often impact the private life and expectations of any employee and translate into motivation and performance.²⁸ Because teams are virtualized and dispersed, care must be taken to instrument the teams with regular group meetings and recognition events and to encourage casual networking. Formal and informal mentors should be assigned to employees. The EDT could assist in helping connect distributed employees and provide team unity.

   In the HSB, the service agents, rather than instructing and monitoring individuals, are focused on the overall team orchestration. The employee responds to business operation requests which are delivered through the COB. These requests follow a defined format, process rules, and take into account the employee's current workload. This impersonal arrangement may not be motivating, and new incentives must be designed to encourage top performance. These could include adequate praise from senior executives based on quality of delivered work, explicit recognition by peers and business-unit service entities, and monetary awards.

2. Designing incentives—Buckingham²⁹ and Keirsey²² suggest that individual personality types are motivated by distinct and predictable incentives. For example, an intuitive thinker (in Keirsey's terminology) strives for recognition by peers and fame as a solid solution provider, whereas a “guardian” type expects to be honored for adhering to all rules and most efficiently performing given tasks. The latter may calculate his or her share of the revenue from any project, while the intuitive thinker prefers to focus on the amount of public recognition. Designing incentives to match the individual's personality type would provide substantial motivation. Nevertheless, caution must be exercised not to breach the privacy of employees.

3. Matching roles and skills to services—Belbin's team roles^30,31 and accompanying research established that an “A-team” does not necessarily turn out the best performance; rather a certain mixture of team roles provides for the best outcome. Transferring this finding to the HSB means that there will be various team members called together for any given project. These will not be just the people with the appropriate skills and experiences, but rather those who fit the team-role profiles. This emphasizes the importance of assessing team performance in SOA separately from individual performance.

   In order to assure that this happens, any individual professional is asked not simply for his or her skills, knowledge, and experiences, but also for the team role type and individual talents which determine the person who will perform best. One is more strongly motivated by doing what one likes than by what one is forced to do. The extensive surveys done by the Gallup Organization and referenced by Buckingham²⁹ show that the distribution of people who like to take tasks which may look ugly and boring to others is not insignificant; there are always people who prefer to perform what is considered a burden for others.

   Through the employee ADT, the optimal team can be assembled, based on team roles and talents in addition to knowledge, skills, and experience. This technique, once fully implemented, should drive significant overall performance improvement. It is necessary that the change planners understand and implement the most meaningful incentives for the targeted populations.

4. Reinforcing new working styles—New ideas do not sell simply because they are new. Repeated studies in product life cycles show the majority of employees will not move from a given product unless forced. The characteristics of a successful change program have been widely documented: the behaviors must be well understood and required, supported by strong tools, culturally encouraged, and provided with sound business incentives.³ Strong processes and architecture are also important cultural elements.³² Gartner research indicates that success depends on supportive policies and organizational structure.³³

5. Managing individuals—Confidence in the skills of employees and performance reporting are critical for an SOA business to be effective. It is necessary to establish a system that is event-triggering so that only exceptions are presented for handling. Governance is then automated to detect situations that require action; this is precisely how the HSB operates. The manager is freed from a strictly controlling role and instead takes a guiding, advisory, coaching, and moderating role.^34,35

6. Designing measurement metrics—Quality assurance for services in the context of the HSB centers on monitoring. Monitoring ensures that policies are set by administrators. It is important to define a scale that measures the value added by any service. Using this scale, the enterprise has to establish a mechanism that fairly measures the share of each service allocated to any project. The rating meta-data provided by the service delivery tool in the bus provides performance data on particular services and thus on the individual performers.

7. Designing appraisal and compensation metrics—The topic of appraisal and compensation is quite sensitive. Every employee working for an enterprise deserves fair and adequate treatment, reflecting the value of his or her contribution. New ways to judge individual contributions to the enterprise have to be developed, especially when services are requested and delivered through the HSB in a way that is not strictly predefined, without having an item going through a manager who assigns and rates the work item. In most European countries, law requires the workers' councils to be involved and to approve the system for distributing profits among the employees.

Changes to culture and individual behaviors are extensive when SOA is implemented. Careful planning of the change is a critical success factor for the new environment. We next describe a project that IBM conducted as it prepared to implement asset reuse. While the study applied specifically to the reuse of IT components, the methodology is applicable to the entire range of reuse implemented in SOA, including human assets, as well.

The purpose of this project was to prepare for the broad-based participation of IBM's internal development communities in the reuse aspects of ECBA by identifying required cultural and behavioral changes and obtaining approval (“buy-in”) from the executive stakeholders to support the required changes. This approach differed from earlier reuse projects in two key areas. First, reuse was not a stand-alone or an add-on to the project; instead, it was integral to and supported by ECBA. (Barnett³⁶ points out the importance of this integration of reuse into the mainstream architecture and processes of the development environment.) Second, cultural and behavioral changes were studied in advance of the program rollout rather than as a midcourse correction, which had been the more typical approach. An early start on the behavioral and cultural implications of the program was expected to result in faster acceptance and adoption of the program, and thus earlier realization of the anticipated benefits.

The study began with a two-part hypothesis that the root cause of lack of pervasive reuse in earlier programs was associated with a lack of clarity and buy-in by the required participants and a lack of alignment of the reuse goals with other goals and incentives of the organization and the participants. Interviews, workshops, and work sessions were conducted with a core team of subject matter experts and an executive sponsorship team.

In these sessions, perceptions of prior reuse programs along several dimensions were explored: executive support, understanding, inhibitors, enablers, motivators, incentives, and metrics. The hypotheses were validated, and specific issues were identified. The two most prominent issues belonged to the categories of funding and risk. The funding issue related to the increased cost of developing IT components as compared to developing code for a single use and the lack of clarity in the financial model for IT component use and maintenance. The risk issue arose from uncertainty about the end-to-end process for IT component development, maintenance, and use arrangements. These issues served as a foundation for the next step, which was to identify for each job role how culture and behaviors would have to change for the program to succeed.

SOA requires activities of specific job roles at key steps in the business strategy, IT strategy, and solution development process that are quite different from the pre-SOA environment. Detailing these new activities is essential to success. For example, the IT executive must champion SOA through targeted communications, by setting specific objectives for the organization, and by reviewing results in an ongoing manner.

The next step was to specify the behaviors required of each job role at key points in the development life cycle and to contrast them to current behaviors. This comparison of current to desired behaviors enabled a difficulty assessment of the change to be performed by each individual in the organization. Extreme difficulty was most often related to conflicting goals for a particular individual. For example, if a developer was encouraged to minimize risk to the project and the reusable asset was assumed to carry additional risk, the developer would not be willing to adopt the reuse behaviors. In this case, either the goal of minimizing risk had to be reconciled explicitly with the goal of reuse, or the perceived risk in the use of IT components had to be reduced.

Degree of difficulty can also be related to barriers in the organization or in its processes that inhibit the adoption of the new behaviors. For example, if it were difficult to locate reusable assets, the architect would not be willing to commit to the new behavior. Another dimension considered for the behaviors was the importance of the behavior to the overall success of the program.

The process for working through this exercise with the subject matter experts was straightforward. Within each of the life-cycle stages, the role of each participating job holder was described. The role was then translated to behaviors by using the following three-step methodology in a facilitated workshop setting:

1. Envision and design the target behaviors—At a high level, the group defined and reached consensus regarding ideal behaviors related to success of the reuse program.
2. Define and size the gaps—The participants then contrasted the current organizational culture and behaviors to the target culture and behaviors and assessed the difficulty of making the transition.
3. Prioritize target behaviors—Finally, the participants assessed the relative importance of achieving each targeted behavior based on its expected contribution to the success of the reuse program.

With the complete list of prioritized behaviors, the next step was to identify potential motivators, incentives, enablers, and metrics for the new behaviors.

Perhaps the biggest failure in most organizations that have attempted transformations is the lack of incentives for the desired behaviors.³ Although formal performance measures related to rewards serve as powerful incentives for reuse, there are others as well. By working with the subject matter experts, several general motivators and incentives were identified for asset reuse. For example, SOA's standing as a leading-edge technology was a motivator for the development community, with the promise of new skills, the ability to expand one's network, and the potential to influence the direction of strategic initiatives. Accentuating technological innovation would therefore be an important aspect of the communication for this program. Another example of a motivator is the potential for recognition for those who author reusable assets. Monetary awards for individuals or teams may be considered, especially for early adopters. Drawing attention to the program in these positive ways serves to advance understanding, interest, and acceptance of the program and to take advantage of the variety of motivators cited above.

With the desired behaviors as a starting point, the subject matter experts identified the enablers that would support the new behaviors. Some enablers were of a general nature and were critical success factors for all change programs; these included the urgent case for change, executive sponsorship, targeted training, a robust communication program, and metrics that demonstrated the business value of the change. Other enablers were unique to this particular project and included reconciliation of conflicting individual goals, infrastructure support, a management system that included an escalation process for issues, a funding model that encouraged participation, integration with existing development processes, and the formation of a knowledge community.

Another key element of the rollout program was the development of metrics that provided ongoing indicators of the success of the program as well as data to assess the participation of individuals. Developing appropriate metrics for reuse, while challenging, is the only way to build a meaningful business case.³⁶ Such metrics are available through the meta-data described in the section “Imperative for a new organizational structure.” Metrics should be used in the formal performance management system to describe the expectations for individuals pertaining to their roles in SOA. This technique is aligned with the four-step approach recommended by Buckingham and Clifton²⁹ to change behavior: determine how to measure the desired performance, build a scorecard for every employee, conduct manager/employee discussions, and monitor progress. The power of this technique stems from the fact that essentially all human-resource actions rely on the results of these more formal commitments and thus receive immediate and ongoing focus.³⁷

The final step was to plan for the rollout of a comprehensive cultural and behavioral program, including a communication program to reach the targeted populations on an ongoing basis. Communication included educational materials on SOA, how it works, and the anticipated business benefits. Executive sponsorship was a key communication element, and the plan included coverage of this topic whenever executives communicated with the organization. Collaboration tools were essential for success and figured prominently in the communication.

Champions for SOA were identified, and they initiated communication on an ongoing basis that included education and results, especially success stories. Measures of success for the program were identified, monitored, and reported regularly. Three to four months after the rollout, an assessment of the success of the program was used to identify whether midcourse corrections were required. The assessment also provided feedback on issues and emerging best practices. This closed-loop feedback process enabled the fastest implementation of the desired culture and behaviors leading to successful reuse, the key element of SOA.

In this paper, we showed how SOA can be leveraged beyond IT transformation to meet a corporation's on demand needs in the 21st century. For realizing the true value of SOA, organizational and cultural transformation is essential. While change is always accompanied by uncertainty, the potential benefits that can be realized outweigh those risks.

The empirical results from IBM's business transformation initiatives are still being gathered, and the initial results have been promising. As more corporations embrace the on demand vision and the accompanying transformations described here, experience will provide a broader set of outcomes, lessons, and pitfalls. Analyzing the spectrum of these results will help derive more prescriptive and detailed guidance.

*Trademark, service mark, or registered trademark of International Business Machines Corporation.

Accepted for publication June 6, 2005; Published online October 19, 2005.