Soa Governance
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Soa Governance as PDF for free.
More details
- Words: 7,057
- Pages: 21
SOA Governance Enabling Sustainable Success with SOA October 2006
TABLE OF CONTENTS THE SOA GOVERNANCE IMPERATIVE .................................................................................. 3 SOA GOVERNANCE DEFINED ................................................................................................ 3 THE CASE FOR SOA GOVERNANCE ....................................................................................... 4 ARCHITECTURE GOVERNANCE ............................................................................................... 5 SERVICE LIFECYCLE GOVERNANCE ........................................................................................ 6 Design-Time Governance ............................................................................................... 7 Run-Time Governance.................................................................................................... 7 Change-Time Governance.............................................................................................. 8 TECHNOLOGIES BEHIND SOA GOVERNANCE................................................................... 10 REGISTRY ........................................................................................................................... 11 REPOSITORY ...................................................................................................................... 11 Advantages of an Integrated Registry/Repository ........................................................ 12 POLICY ENFORCEMENT POINTS ........................................................................................... 13 Design-Time Enforcement: Registry/Repository........................................................... 13 Run-Time Enforcement: Message Transport................................................................ 14 Change-Time Enforcement: IT Management System .................................................. 16 GOVERNANCE RULES ENGINE ............................................................................................. 16 LIFECYCLE MANAGEMENT.................................................................................................... 17 GETTING STARTED WITH GOVERNANCE ........................................................................... 18 APPENDIX: SOA GOVERNANCE REQUIREMENTS CHECKLIST ....................................... 19
©2006 webMethods, Inc. All rights reserved.
Page 2
THE SOA GOVERNANCE IMPERATIVE Many companies are still in the early stages of SOA adoption and so the practice of SOA governance—and likely the concept itself —will be new territory for many IT professionals. And yet, if companies are to realize any meaningful and lasting impact from SOA, then governance is a fact of life that enterprises are going to have to become comfortable with. More than any other factor over the long term, governance will make the difference between SOA success and failure, and proficiency in governing the SOA environment will distinguish IT leaders from laggards. So what exactly is SOA governance, why is it important, and how do you make it real? This white paper offers a practical explanation of SOA governance, where governance should be applied, and the technologies to consider when putting an SOA governance system into place. By making governance an up-front part of your SOA implementation strategy, you will greatly enhance your chances of success and achieving a lasting return on your SOA investments. “In 2006, lack of working governance mechanisms in midsize-to-large (greater than 50 services) post-pilot SOA projects will be the most common reason for project failure. (0.8 probability)” Gartner, Management Update: Predicts 2006: The Strategic Impact of SOA Broadens Jess Thompson, et al.
SOA Governance Defined Governance: The art and discipline of managing outcomes consistent with measurable preconditions and expectations through structured relationships, procedures, and policies applied to the organization and utilization of distributed capabilities that may be under the control of different ownership domains. While the term may be novel, the act of governance and the underlying rationale will be very familiar to IT. In most organizations, virtually every IT resource and process will have some level of governance associated with it in the form of policies, rules, and controls that define how a particular asset is managed and utilized, or parameters around how a certain IT function is performed. For example, IT management will have established policies for how projects are initiated, funded, prioritized, and staffed. An enterprise application package such as SAP or Siebel will have rules and operational procedures concerning issues such as how change requests are processed and approved, which employees are allowed access to the system and their level of access, and how new versions of the software are rolled out. Collectively, the act of establishing and enacting these rules falls under the broad umbrella of IT governance, with the purpose being to institutionalize discipline and maturity in IT processes so as to gain greater control and economies.
©2006 webMethods, Inc. All rights reserved.
Page 3
SOA governance, then, is the subset of IT governance related to establishing policies, controls, and enforcement mechanisms—within the context of the activities and constructs associated with SOA—similar to those that exist for managing and controlling other aspects of IT. Initially, the concept of SOA governance was applied narrowly to the development and use of Web services, for example, validating the conformance of a Web service with specific standards or managing Web services in the SOA run-time environment. Today, however, it is generally understood that SOA governance has a broader meaning that spans SOA architecture as well as the governance of services across the entire implementation lifecycle. Before exploring the scope of SOA governance in further detail, it is worth considering why governance is so fundamental to SOA’s success.
The Case for SOA Governance SOA provides organizations with a powerful opportunity to transform the way in which they deploy information technology. This includes benefits for both the IT organization and the line of business. For IT, these benefits include:
Increased standardization of systems and operational practices
An improved ability to consolidate existing computing assets to eliminate redundant functionality while preserving needed features and services
The development of new systems offering higher levels of visibility, control, and compliance
The ability to evolve the systems infrastructure incrementally
For the business, SOA holds the promise of increasing the flexibility of IT resources, improving the visibility and adaptability of business processes, and increasing the organization’s agility and responsiveness to market conditions. By definition, however, enterprise-level SOA means a dramatic increase in the number of interdependent moving parts in the environment. The tradeoff is that this increase in the number of moving parts is accompanied by an exponential increase in the number and complexity of these interdependencies. Each service is an asset that has to be properly designed to be useful within a larger portfolio of business services – versioned, secured, managed, and monitored to ensure that it performs with the expected quality-of-service. IT is accustomed to dealing with these issues on an aggregate level—at the scope of an application package, for instance—but SOA creates a challenge that is an order of magnitude greater by requiring these issues to be addressed at the level of individual services. Further compounding the challenge is the fact that services use and interact with other services, which increases the complexity of change management, testing, and deployment. How do you recreate a network of inter-dependent services in an isolated environment to perform regression testing, for example?
©2006 webMethods, Inc. All rights reserved.
Page 4
This complexity is compounded by the possibility that services may be reused across organizational boundaries and heterogeneous domains of ownership. This causes the inherent requirement for trust and accountability to extend to resources outside of the control of the service consumer. This increases the inherent complexity of SOA governance by creating additional organizational challenges to ensure the alignment of disparate stakeholders in an SOA. Thus success with SOA is highly correlated with an organization’s ability to manage complexity and to develop the necessary maturity and infrastructure—in the form of control and enforcement mechanisms—to maintain order over the SOA environment. Without effective SOA governance, organizations will experience some predictable challenges, including:
A fragile and brittle SOA implementation
Services that cannot easily be reused because they are unknown to developers or because they were not designed with reuse in mind
Lack of trust and confidence in services as enterprise assets, which results in a “build it myself” mentality (further compounding the lack of reuse with redundancy and unnecessary duplication of functionality)
Security breaches that cannot easily be traced
Unpredictable performance
Failure to overcome these challenges will cause a drag on an organization’s continued deployment of SOA and, in all likelihood, result in SOA being discarded as a failure. Fortunately, the majority of organizations exploring SOA recognize the need to address these issues, and companies that are serious about SOA are generally serious about governance. So, having established the importance of governance in making SOA successful, on what fronts is SOA governance required?
Architecture Governance The first requirement of SOA governance is architecture governance. Architecture governance is necessary to ensure that SOA as architecture evolves by design and not by accident. To the extent that it mirrors governance requirements in other areas of IT architecture, SOA architecture governance practices can be adapted from existing Enterprise Architecture processes. These include:
Establishing corporate technology standards
Defining the high-level SOA architecture and topology, as well as the infrastructure capabilities that the SOA should incorporate
Determining the SOA platform strategy and making decisions about particular vendor products and technologies
©2006 webMethods, Inc. All rights reserved.
Page 5
Specifying the management, operations, and quality-of-service—security, reliability, and availability—characteristics of the SOA
Establishing criteria for SOA project design reviews
In addition, a key aspect of SOA architecture governance is defining a roadmap that will guide the smooth and orderly evolution of the architecture over time. The majority of corporate SOA strategies will involve overlaying and transforming the existing systems architecture in stages, rather than a wholesale replacement of the current infrastructure. Governance is needed to ensure that decisions made along the way align in a consistent direction and maintain the coherency of the SOA architecture. SOA architecture governance is a discipline that architects and IT organizations will acquire with relative ease if they have experience in parallel areas of distributed systems architecture, for example, establishing the architecture of an enterprise-wide integration infrastructure. In contrast, when it comes to the second frontier of SOA governance, organizations are likely to encounter certain requirements for the first time.
Service Lifecycle Governance A fundamental underpinning of SOA is that it involves the creation of discrete, welldefined services that exist not only as building blocks of larger systems and applications, but as independent entities. For the first time within the IT environment, SOA exposes standalone application functionality at a fine-grained level of granularity, thus necessitating a new form of governance—service-level lifecycle governance.
N RU
DE SIG
N
MANAGE
SOA GOVERNANCE LIFE-CYCLE AN
US E
PL
Service-level governance applies at the level of individual services and covers a wide gamut of requirements and situations. A useful approach to categorize the scope of activities associated with servicelevel governance is to consider the lifecycle of a service— beginning with its design, to its use in a run-time environment, to ongoing management and change of the service—as well as the constituencies who have a vested interest in the governance of these services.
CHANGE
SOA Governance Lifecycle
©2006 webMethods, Inc. All rights reserved.
Page 6
Design-Time Governance Design-time governance is primarily an IT development function that involves the application of rules for governing the definition and creation of Web services. Policies might include ensuring that services are technically correct and valid, and that they conform to relevant organizational and industry standards. Examples of this type of validation might include checking that a service is compliant with the Web Services Interoperability (WS-I) profiles—usage guidelines that ensure Web services implemented on different platforms are interoperable—by automatically verifying service schemas, validating namespaces, and other such controls. If an organization has an SOA governance infrastructure in place—in the form of software that facilitates the implementation of SOA governance practices—these checks can be invoked automatically when developers check services into a registry. In addition, approval and notification workflows can be triggered by a governance-enabled registry to ensure that services pass through pre-defined review and approval steps so that they meet architectural and organizational standards for business function encapsulation, reusability, reliability, and so on. By ensuring that these reviews are performed by appropriate members of the organization, it becomes possible to manage the quality and coherency of the service portfolio effectively. Design-time governance will be of most concern to business analysts, architects, and developers building services. Key issues to consider include:
Determining the fitness of a service as an enterprise-class asset (where fit is a function of the business functionality that is encapsulated, the likelihood of reuse, and the importance of the service within the overall portfolio of services).
Identifying which services to build against the backlog of business requirements.
Ensuring the strategic design of business services and validating that their interfaces and implementation conform to established design patterns and other corporate standards and practices.
Establishing the governance standards to which different categories of services will be held, understanding that different levels of governance will be appropriate for different classes of services (internal-use vs. services exposed to business partners, for example).
Other capabilities of design-time governance include fine-grained access control over assets in the registry, so that only authorized users are able to publish, search, and view services. In addition, the ability to label services and classify providers and consumers makes it possible to have some services visible to certain classes of service consumers and not others, a feature that is particularly important for partitioning access in a shared services model.
Run-Time Governance Run-time governance is primarily of interest to IT operations. Governance at run-time revolves around the definition and enforcement of policies for controlling the deployment, ©2006 webMethods, Inc. All rights reserved.
Page 7
utilization, and operation of deployed services. These run-time policies typically relate to non-functional requirements such as trust enablement, quality-of-service management, and compliance validation. Examples of run-time governance include:
Checking a service against a set of rules before it is deployed into production, for example, to ensure that only a certain message transport or specific schemas are used
Securing services so that they are accessible only to authorized consumers possessing the appropriate permissions, and that data is encrypted if required
Validating that services operate in compliance with prescribed corporate standards, in effect, to confirm that a service is not just designed to be compliant, but that its implementation is actually compliant
A more specific case of run-time governance involves service-level monitoring and reporting. In order for the run-time SOA infrastructure to assess whether a given service is performing at the required level for a given consumer—in terms of response time, throughput, and availability—it is necessary to have an explicitly defined service level agreement (SLA) between the service consumer and provider. SLAs can be expressed in terms of service contracts between consumer-provider pairs, and they establish the reference points for compliance monitoring and reporting by the SOA run-time environment. By tracking the actual performance of a service and comparing it to the requirements specified in the SLA, the system can identify non-compliant services and prompt remedial action (for example, automatically instantiating another instance of the service to improve load-balancing or alerting operations staff).
Change-Time Governance Change is inevitable and, at some point, services deployed in the run-time environment will have to be changed to adapt to new business requirements. Since the majority of services will be designed once and then modified several times over their lifespans, change-time governance—the act of managing services through the cycle of change—is arguably more important in the long term than design-time governance. An essential dimension of change-time governance is the nature of the changes to the behavior of the SOA system. These are •
Customization
•
Composition
•
Configuration
Traditional software development is only changed through recoding and deploying new code, which we refer to here as customization. With SOA, two additional change mechanisms are enabled. Most SOA practitioners are familiar with composition, which is a style of development which does not emphasize the creation of new code, but of combining pre-existing deployed components. The most rapidly changing aspects of SOA system behaviors are encoded in metadata configurations such as processes, contracts, and ©2006 webMethods, Inc. All rights reserved.
Page 8
policies. SOA enables the externalization of the fastest changing parts of your SOA into metadata. Since processes, contracts, and policies all influence the behavior of deployed systems, changing SOA systems through configuration can be an extremely rapid form of change. Because of the rapidity of change enabled by SOA, appropriate controls are even more important to ensure managed outcomes. Change-time governance requirements and considerations include:
Understanding inter-service relationships and dependencies
Performing impact analysis to determine the implications of changing a particular service within the run-time environment
Managing the rollout of services into the existing run-time environment
Managing service custody transfers through the design, coding, testing, and deployment stages
Managing changes to existing policies and service level agreements
An important aspect of change-time governance is involvement from the line of business. While easy to overlook when looking at governance from an IT-centric perspective, this need arises from the fact that services exist to support business functions as well as the inter-organizational relationships and dependencies that are implicit in SOA, particularly when services are exposed and invoked across organizational and corporate boundaries. Since changes are generally initiated and driven by business requirements, business users need to be intrinsic participants in the governance lifecycle. Consider, for example, a service that enables vendor managed inventory. A change in the service, say, to reduce inventory data latency from a week to one day, will involve not only technical changes to the service (and possibly source applications and databases), but, more importantly, changes in the business relationship between the company and its suppliers. A comprehensive governance strategy is needed to ensure coordination between the technical and business-level changes. As with design- and run-time, this change-time governance can be facilitated by a governance-enabled SOA infrastructure that allows change-time policies to be defined and enforced through service contracts and workflows.
©2006 webMethods, Inc. All rights reserved.
Page 9
TECHNOLOGIES BEHIND SOA GOVERNANCE While SOA governance is not a shrink-wrapped capability that you can implement off the shelf without also addressing organizational and procedural issues, at its foundation is the ability to enforce and automate policies across the service lifecycle. This creates the opportunity for software mechanisms that enable the automation and enforcement of governance policies. This section of the white paper outlines the key moving parts of an SOA governance system which performs such policy-related functions.
Stakeholders
Lifecycle Management
Design Time
Policy Configuration
Change Time
Registry Repository
Governance Rules Engine
Policies
Policy Enforcement Points
Run Time
Registry Repository
Message Transport
Management System
Key Components of an SOA Governance System
At a basic level, an SOA governance system should facilitate service-level governance across the lifecycle from design-time to run-time to change-time. It should allow polices to be defined and created, and provide mechanisms for these policies to be enforced at each phase of the service lifecycle. The main components of this system include:
A registry, which acts as a central catalog of business services
A repository, for storing policies and other metadata related to the governance of the services
Policy enforcement points, which are the agents that enact the actual policy enforcement and control at design-time, run-time, and change-time
A rules engine for managing the declaration of policies and rules and automating their enforcement
An environment for configuring and defining policies and for managing governance workflows across the service lifecycle
©2006 webMethods, Inc. All rights reserved.
Page 10
Registry A registry is usually identified as one of the first requirements of SOA adoption and registries play an important role in governance. In simple terms, a registry is a catalog or index that acts as the “system of record” for the services within an SOA. A registry is not designed to store the services themselves; rather, it indicates their location by reference. Having a centralized catalog of services is significant from an organizational perspective because it enables the easy discovery, reuse, and management of services. An SOA registry typically fulfills the following functions:
Stores service descriptions, information about their end-points (the network resource where the service functionality is implemented), and other technical details that a consumer requires in order to invoke the service, such as protocol bindings and message formats
Allows services to be categorized and organized
Allows users to publish new services into the registry and to browse and search for existing services
Maintains service history, allowing users to see when a service was published or changed
As the place where services are made known within the SOA, a registry is also a natural management and governance point. For example, compliance requirements—such as conformance with the WS-I Basic Profile or the use of specific namespaces and schemas—might be imposed on services before they are allowed to be published in the registry. Or, as services are registered or changed, the registry also has the ability to trigger approval and change notification workflows so that stakeholders are alerted to changes. As such, a robust registry is an important component of any SOA governance solution. Another important factor is the interoperability of the registry with other components of the SOA infrastructure. OASIS provides a platform-independent standard for registry interoperability known as UDDI (Universal Description, Discovery, and Integration). UDDI defines a Web services-based programming interface that allows different consumer applications, tools, and run-time systems to query the registry, discover services, and interact as required to provide management and governance capabilities. While it is not a pre-requisite for an SOA registry to be based on UDDI, it is the most commonly adopted standard and ensures the greatest degree of compatibility with other products in the environment.
Repository While the registry plays a central role in policy enforcement, the registry itself does not provide sufficient context for the breadth of SOA governance requirements described in this white paper. For example, policies—in the form of rules and restrictions that are enforced on services—and consumer/provider service level agreements are generally not constructs that are stored in a registry (for one reason, they are not constructs that are
©2006 webMethods, Inc. All rights reserved.
Page 11
known to UDDI). Thus another data store, usually referred to as a repository, is needed for storing governance-related artifacts and supporting the full complexity of managing service metadata throughout the service life cycle. The term “repository” is used in many different contexts—for example, a data repository (SQL database), a document repository (file system), or a source code repository (version control system)—but in the context of SOA governance, the repository can be thought of as a centrally-managed policy store. Among other things, a governance repository should support the following capabilities:
An information model or taxonomy for representing and storing organizational and regulatory policies that can be translated into rules that are enforced by the SOA governance system. It should be possible for policies and rules to be interpreted by people or machines (and sometimes both) as appropriate.
Audit capabilities for tracking the trail of changes and authorizations applied to assets within the repository context.
Identity management capabilities and role-based access controls to ensure that only appropriate parties have access to policies.
A notification system and content validation capabilities to provide additional assurances that policies are well-formed, consistent, and properly applied.
The requirement for a logically centralized repository is particularly important for codifying and enforcing a single “official” set of policies across the organization. However, the actual repository itself may have a federated architecture for scalability and to enable the use of the repository across different geographic regions, multiple lifecycle constituencies, and across corporate boundaries.
Advantages of an Integrated Registry/Repository While the registry and repository have been discussed as separate concerns, in practice there are many benefits to combining the two functions into a single entity. To function properly together as a system, the registry and repository should maintain a consistent view of service definitions, service versions, consumer and user identities, and other information. Implementing them as separate products creates the burden of duplicate data entry, sets up the need to synchronize information, and increases the risk of inconsistencies between the two. When the registry and repository are unified with a single normalized and standardsbased information model and underlying datastore, the integrity of both governancerelated metadata and the information model can be assured. The unified approach also eases the enforcement of policies that apply across the boundary between the registry and the repository. Standard registry capabilities can be offered by an integrated registry/repository with a UDDI interface to the policy repository that allows access to the relevant data.
©2006 webMethods, Inc. All rights reserved.
Page 12
Policy Enforcement Points Thus far, the role of the registry and repository has been established and the case has been made for an integrated registry/repository to create a consistent policy context. These policies, in turn, need to be enforced. Within an SOA governance system, this function is fulfilled by policy enforcement points. The places where policies are actually applied and enforced—the policy enforcement points—change depending on the lifecycle stage. During design-time, the registry/repository itself is the point of enforcement. During run-time, policies are generally enforced by the underlying message transport system that connects service providers with consumers. Finally, during change-time, policies are typically enforced by the IT management system.
Design-Time Enforcement: Registry/Repository Since the registry/repository is the system of record for both service interfaces as well as the assets, attributes, and metadata associated with them, it provides a logical point at which to enforce policies about the design of these particular artifacts. Design-time policies are typically applied as artifacts—which could include WSDL files, schema definitions, process models, and project documentation—are checked into the registry/repository. The following features are desirable in the design-time policy enforcement point:
Identity management — In order to establish rights and responsibility in the registry/repository it is first necessary to identify users (for example, via an LDAPbased identity system or other directory), service consumers, and other participants. Identity is also important for metering usage, logging for audit purposes, and applying approval requirements and other governance processes on an individual or role basis.
Access control — Coupled with identity management, the system should offer fine-grained access configurations over all aspects of registry/repository assets. This includes the ability to secure assets as well as policies, governance processes, and asset classifications.
Automated notifications and approvals — The ability to trigger events in response to Create, Read, Update, or Delete activities in the registry/repository allows alerts, approval processes, content validation scans, and other actions to be automated. These triggers might be applied either before or after the interaction in question. For example, a policy might be established that a design review approval is needed before an object is created in the registry/repository.
Content validation — Content (in the form of assets that checked into the registry/repository) should be scanned and validated according to their type and preconfigured compliance checks. Common validations include WSDL validation, XML schema validation, testing for namespace violations, schema validation, and other interoperability-related scans. For example, service consumers expect interfaces to be well-formed and interoperable, so the registry/repository should
©2006 webMethods, Inc. All rights reserved.
Page 13
automate the process of scanning and assuring that WSDL documents are wellformed and conformable with WS-I interoperability profiles.
Audit trails — A fundamental capability for establishing accountability is the ability to track interactions between participants and the registry/repository, along with the sequence and details of those activities. This record can be used for governance enforcement “after the fact” and to establish usage patterns for guiding process improvements.
Run-Time Enforcement: Message Transport Run-time policy enforcement relies on an SOA infrastructure that is able to exercise policy enforcement in a way that is transparent to, and independent of, the service providers and consumers. This is achieved through an agent or intermediary that resides between provider and consumer and a registry/repository that addresses both the needs of runtime service discovery as well as policy enforcement. The intermediary interacts with the registry/repository to find services and their run-time policies and enforces the policies during the execution of the service. In an SOA, the run-time system is typically a message transport or mediation layer such as an Enterprise Service Bus (ESB). The message transport brokers transactions between service provider and service consumer and frequently offers additional functions such as data transformation, message queuing, reliable messaging, and other operational capabilities.
Consumer
Message Transport Policies
Provider
Registry/ Repository
Web Service
Run-Time Policy Enforcement via a Message Transport Intermediary
Without SOA, the ability to control and manage applications in this manner is restricted both by the scope and the capabilities of the underlying platform. When different applications are integrated, it is generally infeasible to apply a common policy context to the integrated result. A typical challenge is enforcing access security when two applications with different user communities are integrated. For example, application A automatically ©2006 webMethods, Inc. All rights reserved.
Page 14
draws data from application B, but application A users are not authorized to use application B. How do you now control the data that users have gained access to within application A? With the intermediation provided by the message transport, it becomes possible for a distributed network of services to share a common policy-managed context. This is a powerful capability, which emerges as a direct result of SOA. Since run-time policies are typically applied to messages that flow across the message transport system, the types—and level of sophistication—of run-time policies that can be defined and enforced depend on the capabilities of the underlying intermediary. Desirable areas of policy configuration include the following:
Consumer identification and security — Identifying consumer applications to prevent unauthorized access to services; configuring the security of services at runtime and enforcing policies such as encryption (for message confidentiality), digital signatures (for message integrity and non-repudiation), and logging for tracing and tracking.
Routing rules — Configuring run-time routing rules to address performance, version management, and other operational requirements. Variations include: content-based routing (examining a message’s content to determine where to route it according to specific rules, for example, processing certain types of orders via a different process); version-based routing (to support version management and the deprecation of services); and preferential quality-of-service routing (giving consuming applications different processing priorities based on business priorities and defined service levels).
Transformation rules — Translating between different message transports and technology protocols to facilitate application connectivity, or transforming data between consumer and provider.
Service Level Agreement (SLA) management — Policies for managing performance and availability to match the requirements of an SLA, for example, routing a request to a backup service in the event of a failure of the primary service provider, or balancing the request load across additional back-end service to improve performance.
Logging, monitoring, and alerting — Collecting service-level data and establishing rules based on aggregate counters for response time, throughput, errors, and other transaction data so that alerts can be generated when there are violations to predefined SLAs.
Finally, while the intermediary and registry/repository are logically decoupled, a dependency exists to the extent that the intermediary has to understand and interpret the policies defined in the registry/repository. As such, it is advantageous to have a message transport system and registry/repository that are interoperable out of the box; otherwise, this is an integration issue that the implementer has to address.
©2006 webMethods, Inc. All rights reserved.
Page 15
Change-Time Enforcement: IT Management System Whereas design- and run-time policy have “hard” gates – approval for access to the registry/repository and connectivity via the message transport – change-time enforcement relies to a greater extent on IT change management practices and procedures than on enforced control points. Nevertheless, the IT management system—collectively, the set of systems management software tools and services that the IT organizations uses to manage, monitor, and control their business applications and software infrastructure— and the registry/repository combine to play an important role in change-time governance. Unlike previous software paradigms where an application package enters a support or maintenance phase once put into production, SOA involves a dynamic network of interdependent services that are in an ongoing state of adaptation and optimization. Since services, transactions, and SOA events of interest can be monitored by the IT management system, it is a logical source of run-time information that can be fed back into the registry/repository to facilitate the orderly evolution of the SOA environment. This information might include:
SLA-related metrics, such as the average response time, availability, or throughput of a specific service
Process-related metrics in the form of Key Performance Indicators (KPIs), which associate services with user-defined business process metrics (e.g., average order amount)
Business activity monitoring (BAM), alerting, and notification events related to business-level exceptions.
Information such as this can be used to optimize service delivery during the change-time cycle by guiding adjustments in policies, service levels, or in the services themselves. Changes to services will require the change-time governance practices described earlier to be put into effect, for example, performing an impact analysis to assess the implications of changing a service and dealing with the resulting version management issues. As with integration between the message transport and the registry/repository, it is beneficial to have out-the-box linkages between the registry/repository and the management system so that data flows seamlessly between the two without the need for additional integration.
Governance Rules Engine A rules engine is not strictly a requirement of an SOA governance system, but incorporating rules engine technology within the registry/repository enables a significant degree of flexibility and automation, while reducing the reliance on humans to perform mechanical governance tasks (and the associated risk of error). Rules are typically associated with events, while the rules engine handles the firing and chaining of rules. For example, a company’s policy might be that access to services in production is strictly limited to staff belonging to an IT operations role, whereas in the test
©2006 webMethods, Inc. All rights reserved.
Page 16
environment, access is also granted to developers. The rules engine could automate the process of setting and resetting access control switches at lifecycle milestones such as when a service is promoted from development into testing or production. A rules engine also provides the basis for creating complex policies based on reusable templates. In addition to automating governance tasks, the rules engine can also help to deal with policy federation, or the ability to allow multiple policy authors and authorities. This is an important use case for enterprise SOA adoption where governance policies might not be authored and controlled by a single department or organization. A more robust model— which is the basis for policy federation—is to enable both centralized as well as distributed policy creation. Policy federation requires the establishment of guidelines and rules for reconciling policies that come into conflict, and the rules engine assists in the execution of these rules.
Lifecycle Management The final key ingredient of an SOA governance system is the user environment that presents the human interface to the registry/repository and which incorporates the governance lifecycle processes and workflows. Typically, the process workflow includes the following steps:
Publishing of a service by an authorized provider
Discovery of a service by a potential consumer
Requesting use of the service by the consumer
Agreeing on the terms of delivery of the service
Authorizing the consumer
Provisioning of the service
Monitoring of the service delivery
Related to each of these steps, organizations might define approval and notification workflows, exception alerts, and a variety of other process steps. The SOA governance lifecycle management capability will manifest itself in the forms of screens—with information customized to the user’s role within the governance framework—reports, and integration with e-mail to communicate notifications and approval requests. An important feature when services are extended to business partners is the ability to make these lifecycle management capabilities available securely across the firewall. To summarize, an effective SOA governance system should take a comprehensive approach that spans the service lifecycle from design-time to run-time to change-time. It should address the needs of the different roles involved in an SOA and enforce the governance workflows between these roles. Finally, the design of the governance system itself should build on the value and benefits of the loosely-coupled nature of SOA.
©2006 webMethods, Inc. All rights reserved.
Page 17
GETTING STARTED WITH GOVERNANCE Ultimately, SOA governance is about maintaining control over, and having visibility into, the SOA environment in order to engender the level of trust required for ongoing adoption and success. Since many companies are just getting started with SOA, a question that often comes to mind is how to get started and at what point of SOA adoption should governance become a consideration? While effective governance rests in how it is institutionalized within the organization, the key to getting started is to take a strategic approach and to address the “big picture” governance requirements first. Specifically: 1) Make an SOA governance strategy a “must have” component of your broader SOA strategy. Ensure that governance capability-related milestones are synchronized with SOA adoption milestones so that you do not end up trying to retro-fit governance after the fact. Ideally, the right time for governance is before you put any services into place so that any SOA pilot proves out not only the approach itself, but also the related governance practices along the way. 2) Establish a governance roadmap. As part of the SOA governance strategy, there should be a roadmap that defines which specific governance capabilities your organization needs or wants to put into place, and how they will be implemented. 3) Start at the beginning. Generally, organizations will first want to pay attention to the SOA architecture and to design-time governance policies in order to get the SOA journey off on the right foot. In fact, since architecture and governance are what separate a collection of Web services from being a true SOA, organizations that have gone down the path of simply developing and exposing Web services without the appropriate controls or a broader architecture will want to consider an investment in governance. 4) Put a governance system into place. While governance is not a solution that comes in a box, having the right technology framework makes it easier—and in some situations is the only scalable way—to enforce policies and controls. As explained in this white paper, this framework should include mechanisms for defining policies and service contracts and enforcing them through the service lifecycle through workflows, intermediaries, and other automated means. By establishing the right balance between strategy, organizational practices, and supporting technologies, companies will be able to turn the concept of SOA governance into a practical reality and build a lasting foundation for SOA success.
©2006 webMethods, Inc. All rights reserved.
Page 18
APPENDIX: SOA GOVERNANCE REQUIREMENTS CHECKLIST The following checklist identifies key technical and functional requirements of an SOA governance solution and can be used to assess the completeness of your governance implementation strategy.
Registry/Repository Service metadata typing and validation Service relationship and dependency management Search by name, description, keywords, attributes Advanced search (multi-keys, Boolean expressions) Storage of business-level metadata Organization and business unit management Metadata filtering Delegated administration Consolidated auditing on registry/repository lifecycle actions File-system and database persistence options
Service Access Workflow-driven approval/notification request processes User-configurable policies and assertions Service request reporting Consumer data collection
Service Publishing Workflow-driven approval/notification processes User-configurable policies and assertions WSDL validation and conformance reporting Customizable user-defined validation policies (e.g. corporate policies) Command-line publishing utilities Publication wizards
Service Delivery Consumer/provider pair binding Service delivery contract model Contract enforcement, versioning, deployment, monitoring, reporting SLA management Security terms management Request/response routing management Failover/load-balancing management Logging and monitoring management Policy Deployment Management Runtime metrics warehousing
Service Change Management Service subscription management Service metadata subscription ©2006 webMethods, Inc. All rights reserved.
Page 19
Email and SOAP notifications Synchronous/asynchronous notifications
Federation/Replication Master/slave replication Selective synchronization of multiple repositories Selective promotion of objects across repositories
Management Features Advanced policies and account management User activity auditing Taxonomy management API User Management API Access Control API Subscription API Administration API JAXR API
Security Features Fixed and flexible roles Role-based privileges Service | Taxonomy | Attribute | Repository Object Access by Role Access Control Lists (ACLs) Digital signature support LDAP support
Standards Interoperability UDDI v2 and v3 Handling of any URI data types JSR 93: Java API for XML Registries JSR 223: Scripting for Java JSR 94: Java Rule Engine API
©2006 webMethods, Inc. All rights reserved.
Page 20
ABOUT WEBMETHODS, INC. webMethods provides total business integration to the world’s largest corporations and government agencies. webMethods’ flagship product suite, webMethods Fabric, is the only integrated platform to deliver both SOA and BPM, delivering rapid ROI to our 1,400 customers around the globe. With webMethods, customers can take a process-centric approach to their business problems, allowing them to leverage their existing IT assets, dramatically improve business process productivity and ROI, and rapidly create competitive advantage by making their business processes work harder for their company. webMethods (NASDAQ: WEBM) is headquartered in Fairfax, VA, and has offices throughout the United States, Europe, Asia Pacific and Japan.
Copyright © 2006 webMethods, Inc. All rights reserved. Trademarks The webMethods logo, Get There Faster, Smart Services and Smart Processes are trademarks or registered trademarks of webMethods, Inc.
Worldwide Headquarters 3877 Fairfax Ridge Road South Tower Fairfax, VA 22030 USA Tel: 703 460 2500 Fax: 703 460 2599
US West Coast 432 Lakeside Drive Sunnyvale, CA 94088 USA Tel: 408 962 5000 Fax: 408 962 5329
European Headquarters Barons Court 20 The Avenue Egham, Surrey TW20 9AU United Kingdom Tel: 44 0 1784 221700 Fax: 44 0 1784 221701
Other product names used herein may be trademarks or registered trademarks of webMethods or other companies. Statement of Conditions webMethods, INC. PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OR CONDITIONS OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL WEBMETHODS BE LIABLE FOR ANY LOSS OF PROFITS, LOSS OF BUSINESS, LOSS OF USE OR DATA INTERRUPTION OF BUSINESS, OR FOR INDIRECT, SPECIAL, PUNITIVE, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY KIND, EVEN IF WEBMETHODS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES ARISING FROM ANY DEFECT OR ERROR IN THIS PUBLICATION OR IN THE WEBMETHODS SOFTWARE.
Asia-Pacific Headquarters 6 Temasek Boulevard, #24-01 Suntec Tower Four 15 Blue Street Singapore 038986 Tel: +65 6389 3200 Fax: +65 6389 3299
www.webMethods.com
webMethods, Inc. may revise this publication from time to time without notice. Some states or jurisdictions do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you. All rights reserved. No part of this work covered by copyright herein may be reproduced in any form or by any means—graphic, electronic or mechanical— including photocopying, recording, taping, or storage in an information retrieval system, without prior written permission of the copyright owner.
RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the U.S. government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 (October 1988) and FAR 52.227-19 (June 1987).
©2006 webMethods, Inc. All rights reserved. The webMethods logo and Get There Faster are trademarks or registered trademarks of webMethods, Inc. All other names mentioned are trademarks, registered trademarks or service marks of their respective companies.
TABLE OF CONTENTS THE SOA GOVERNANCE IMPERATIVE .................................................................................. 3 SOA GOVERNANCE DEFINED ................................................................................................ 3 THE CASE FOR SOA GOVERNANCE ....................................................................................... 4 ARCHITECTURE GOVERNANCE ............................................................................................... 5 SERVICE LIFECYCLE GOVERNANCE ........................................................................................ 6 Design-Time Governance ............................................................................................... 7 Run-Time Governance.................................................................................................... 7 Change-Time Governance.............................................................................................. 8 TECHNOLOGIES BEHIND SOA GOVERNANCE................................................................... 10 REGISTRY ........................................................................................................................... 11 REPOSITORY ...................................................................................................................... 11 Advantages of an Integrated Registry/Repository ........................................................ 12 POLICY ENFORCEMENT POINTS ........................................................................................... 13 Design-Time Enforcement: Registry/Repository........................................................... 13 Run-Time Enforcement: Message Transport................................................................ 14 Change-Time Enforcement: IT Management System .................................................. 16 GOVERNANCE RULES ENGINE ............................................................................................. 16 LIFECYCLE MANAGEMENT.................................................................................................... 17 GETTING STARTED WITH GOVERNANCE ........................................................................... 18 APPENDIX: SOA GOVERNANCE REQUIREMENTS CHECKLIST ....................................... 19
©2006 webMethods, Inc. All rights reserved.
Page 2
THE SOA GOVERNANCE IMPERATIVE Many companies are still in the early stages of SOA adoption and so the practice of SOA governance—and likely the concept itself —will be new territory for many IT professionals. And yet, if companies are to realize any meaningful and lasting impact from SOA, then governance is a fact of life that enterprises are going to have to become comfortable with. More than any other factor over the long term, governance will make the difference between SOA success and failure, and proficiency in governing the SOA environment will distinguish IT leaders from laggards. So what exactly is SOA governance, why is it important, and how do you make it real? This white paper offers a practical explanation of SOA governance, where governance should be applied, and the technologies to consider when putting an SOA governance system into place. By making governance an up-front part of your SOA implementation strategy, you will greatly enhance your chances of success and achieving a lasting return on your SOA investments. “In 2006, lack of working governance mechanisms in midsize-to-large (greater than 50 services) post-pilot SOA projects will be the most common reason for project failure. (0.8 probability)” Gartner, Management Update: Predicts 2006: The Strategic Impact of SOA Broadens Jess Thompson, et al.
SOA Governance Defined Governance: The art and discipline of managing outcomes consistent with measurable preconditions and expectations through structured relationships, procedures, and policies applied to the organization and utilization of distributed capabilities that may be under the control of different ownership domains. While the term may be novel, the act of governance and the underlying rationale will be very familiar to IT. In most organizations, virtually every IT resource and process will have some level of governance associated with it in the form of policies, rules, and controls that define how a particular asset is managed and utilized, or parameters around how a certain IT function is performed. For example, IT management will have established policies for how projects are initiated, funded, prioritized, and staffed. An enterprise application package such as SAP or Siebel will have rules and operational procedures concerning issues such as how change requests are processed and approved, which employees are allowed access to the system and their level of access, and how new versions of the software are rolled out. Collectively, the act of establishing and enacting these rules falls under the broad umbrella of IT governance, with the purpose being to institutionalize discipline and maturity in IT processes so as to gain greater control and economies.
©2006 webMethods, Inc. All rights reserved.
Page 3
SOA governance, then, is the subset of IT governance related to establishing policies, controls, and enforcement mechanisms—within the context of the activities and constructs associated with SOA—similar to those that exist for managing and controlling other aspects of IT. Initially, the concept of SOA governance was applied narrowly to the development and use of Web services, for example, validating the conformance of a Web service with specific standards or managing Web services in the SOA run-time environment. Today, however, it is generally understood that SOA governance has a broader meaning that spans SOA architecture as well as the governance of services across the entire implementation lifecycle. Before exploring the scope of SOA governance in further detail, it is worth considering why governance is so fundamental to SOA’s success.
The Case for SOA Governance SOA provides organizations with a powerful opportunity to transform the way in which they deploy information technology. This includes benefits for both the IT organization and the line of business. For IT, these benefits include:
Increased standardization of systems and operational practices
An improved ability to consolidate existing computing assets to eliminate redundant functionality while preserving needed features and services
The development of new systems offering higher levels of visibility, control, and compliance
The ability to evolve the systems infrastructure incrementally
For the business, SOA holds the promise of increasing the flexibility of IT resources, improving the visibility and adaptability of business processes, and increasing the organization’s agility and responsiveness to market conditions. By definition, however, enterprise-level SOA means a dramatic increase in the number of interdependent moving parts in the environment. The tradeoff is that this increase in the number of moving parts is accompanied by an exponential increase in the number and complexity of these interdependencies. Each service is an asset that has to be properly designed to be useful within a larger portfolio of business services – versioned, secured, managed, and monitored to ensure that it performs with the expected quality-of-service. IT is accustomed to dealing with these issues on an aggregate level—at the scope of an application package, for instance—but SOA creates a challenge that is an order of magnitude greater by requiring these issues to be addressed at the level of individual services. Further compounding the challenge is the fact that services use and interact with other services, which increases the complexity of change management, testing, and deployment. How do you recreate a network of inter-dependent services in an isolated environment to perform regression testing, for example?
©2006 webMethods, Inc. All rights reserved.
Page 4
This complexity is compounded by the possibility that services may be reused across organizational boundaries and heterogeneous domains of ownership. This causes the inherent requirement for trust and accountability to extend to resources outside of the control of the service consumer. This increases the inherent complexity of SOA governance by creating additional organizational challenges to ensure the alignment of disparate stakeholders in an SOA. Thus success with SOA is highly correlated with an organization’s ability to manage complexity and to develop the necessary maturity and infrastructure—in the form of control and enforcement mechanisms—to maintain order over the SOA environment. Without effective SOA governance, organizations will experience some predictable challenges, including:
A fragile and brittle SOA implementation
Services that cannot easily be reused because they are unknown to developers or because they were not designed with reuse in mind
Lack of trust and confidence in services as enterprise assets, which results in a “build it myself” mentality (further compounding the lack of reuse with redundancy and unnecessary duplication of functionality)
Security breaches that cannot easily be traced
Unpredictable performance
Failure to overcome these challenges will cause a drag on an organization’s continued deployment of SOA and, in all likelihood, result in SOA being discarded as a failure. Fortunately, the majority of organizations exploring SOA recognize the need to address these issues, and companies that are serious about SOA are generally serious about governance. So, having established the importance of governance in making SOA successful, on what fronts is SOA governance required?
Architecture Governance The first requirement of SOA governance is architecture governance. Architecture governance is necessary to ensure that SOA as architecture evolves by design and not by accident. To the extent that it mirrors governance requirements in other areas of IT architecture, SOA architecture governance practices can be adapted from existing Enterprise Architecture processes. These include:
Establishing corporate technology standards
Defining the high-level SOA architecture and topology, as well as the infrastructure capabilities that the SOA should incorporate
Determining the SOA platform strategy and making decisions about particular vendor products and technologies
©2006 webMethods, Inc. All rights reserved.
Page 5
Specifying the management, operations, and quality-of-service—security, reliability, and availability—characteristics of the SOA
Establishing criteria for SOA project design reviews
In addition, a key aspect of SOA architecture governance is defining a roadmap that will guide the smooth and orderly evolution of the architecture over time. The majority of corporate SOA strategies will involve overlaying and transforming the existing systems architecture in stages, rather than a wholesale replacement of the current infrastructure. Governance is needed to ensure that decisions made along the way align in a consistent direction and maintain the coherency of the SOA architecture. SOA architecture governance is a discipline that architects and IT organizations will acquire with relative ease if they have experience in parallel areas of distributed systems architecture, for example, establishing the architecture of an enterprise-wide integration infrastructure. In contrast, when it comes to the second frontier of SOA governance, organizations are likely to encounter certain requirements for the first time.
Service Lifecycle Governance A fundamental underpinning of SOA is that it involves the creation of discrete, welldefined services that exist not only as building blocks of larger systems and applications, but as independent entities. For the first time within the IT environment, SOA exposes standalone application functionality at a fine-grained level of granularity, thus necessitating a new form of governance—service-level lifecycle governance.
N RU
DE SIG
N
MANAGE
SOA GOVERNANCE LIFE-CYCLE AN
US E
PL
Service-level governance applies at the level of individual services and covers a wide gamut of requirements and situations. A useful approach to categorize the scope of activities associated with servicelevel governance is to consider the lifecycle of a service— beginning with its design, to its use in a run-time environment, to ongoing management and change of the service—as well as the constituencies who have a vested interest in the governance of these services.
CHANGE
SOA Governance Lifecycle
©2006 webMethods, Inc. All rights reserved.
Page 6
Design-Time Governance Design-time governance is primarily an IT development function that involves the application of rules for governing the definition and creation of Web services. Policies might include ensuring that services are technically correct and valid, and that they conform to relevant organizational and industry standards. Examples of this type of validation might include checking that a service is compliant with the Web Services Interoperability (WS-I) profiles—usage guidelines that ensure Web services implemented on different platforms are interoperable—by automatically verifying service schemas, validating namespaces, and other such controls. If an organization has an SOA governance infrastructure in place—in the form of software that facilitates the implementation of SOA governance practices—these checks can be invoked automatically when developers check services into a registry. In addition, approval and notification workflows can be triggered by a governance-enabled registry to ensure that services pass through pre-defined review and approval steps so that they meet architectural and organizational standards for business function encapsulation, reusability, reliability, and so on. By ensuring that these reviews are performed by appropriate members of the organization, it becomes possible to manage the quality and coherency of the service portfolio effectively. Design-time governance will be of most concern to business analysts, architects, and developers building services. Key issues to consider include:
Determining the fitness of a service as an enterprise-class asset (where fit is a function of the business functionality that is encapsulated, the likelihood of reuse, and the importance of the service within the overall portfolio of services).
Identifying which services to build against the backlog of business requirements.
Ensuring the strategic design of business services and validating that their interfaces and implementation conform to established design patterns and other corporate standards and practices.
Establishing the governance standards to which different categories of services will be held, understanding that different levels of governance will be appropriate for different classes of services (internal-use vs. services exposed to business partners, for example).
Other capabilities of design-time governance include fine-grained access control over assets in the registry, so that only authorized users are able to publish, search, and view services. In addition, the ability to label services and classify providers and consumers makes it possible to have some services visible to certain classes of service consumers and not others, a feature that is particularly important for partitioning access in a shared services model.
Run-Time Governance Run-time governance is primarily of interest to IT operations. Governance at run-time revolves around the definition and enforcement of policies for controlling the deployment, ©2006 webMethods, Inc. All rights reserved.
Page 7
utilization, and operation of deployed services. These run-time policies typically relate to non-functional requirements such as trust enablement, quality-of-service management, and compliance validation. Examples of run-time governance include:
Checking a service against a set of rules before it is deployed into production, for example, to ensure that only a certain message transport or specific schemas are used
Securing services so that they are accessible only to authorized consumers possessing the appropriate permissions, and that data is encrypted if required
Validating that services operate in compliance with prescribed corporate standards, in effect, to confirm that a service is not just designed to be compliant, but that its implementation is actually compliant
A more specific case of run-time governance involves service-level monitoring and reporting. In order for the run-time SOA infrastructure to assess whether a given service is performing at the required level for a given consumer—in terms of response time, throughput, and availability—it is necessary to have an explicitly defined service level agreement (SLA) between the service consumer and provider. SLAs can be expressed in terms of service contracts between consumer-provider pairs, and they establish the reference points for compliance monitoring and reporting by the SOA run-time environment. By tracking the actual performance of a service and comparing it to the requirements specified in the SLA, the system can identify non-compliant services and prompt remedial action (for example, automatically instantiating another instance of the service to improve load-balancing or alerting operations staff).
Change-Time Governance Change is inevitable and, at some point, services deployed in the run-time environment will have to be changed to adapt to new business requirements. Since the majority of services will be designed once and then modified several times over their lifespans, change-time governance—the act of managing services through the cycle of change—is arguably more important in the long term than design-time governance. An essential dimension of change-time governance is the nature of the changes to the behavior of the SOA system. These are •
Customization
•
Composition
•
Configuration
Traditional software development is only changed through recoding and deploying new code, which we refer to here as customization. With SOA, two additional change mechanisms are enabled. Most SOA practitioners are familiar with composition, which is a style of development which does not emphasize the creation of new code, but of combining pre-existing deployed components. The most rapidly changing aspects of SOA system behaviors are encoded in metadata configurations such as processes, contracts, and ©2006 webMethods, Inc. All rights reserved.
Page 8
policies. SOA enables the externalization of the fastest changing parts of your SOA into metadata. Since processes, contracts, and policies all influence the behavior of deployed systems, changing SOA systems through configuration can be an extremely rapid form of change. Because of the rapidity of change enabled by SOA, appropriate controls are even more important to ensure managed outcomes. Change-time governance requirements and considerations include:
Understanding inter-service relationships and dependencies
Performing impact analysis to determine the implications of changing a particular service within the run-time environment
Managing the rollout of services into the existing run-time environment
Managing service custody transfers through the design, coding, testing, and deployment stages
Managing changes to existing policies and service level agreements
An important aspect of change-time governance is involvement from the line of business. While easy to overlook when looking at governance from an IT-centric perspective, this need arises from the fact that services exist to support business functions as well as the inter-organizational relationships and dependencies that are implicit in SOA, particularly when services are exposed and invoked across organizational and corporate boundaries. Since changes are generally initiated and driven by business requirements, business users need to be intrinsic participants in the governance lifecycle. Consider, for example, a service that enables vendor managed inventory. A change in the service, say, to reduce inventory data latency from a week to one day, will involve not only technical changes to the service (and possibly source applications and databases), but, more importantly, changes in the business relationship between the company and its suppliers. A comprehensive governance strategy is needed to ensure coordination between the technical and business-level changes. As with design- and run-time, this change-time governance can be facilitated by a governance-enabled SOA infrastructure that allows change-time policies to be defined and enforced through service contracts and workflows.
©2006 webMethods, Inc. All rights reserved.
Page 9
TECHNOLOGIES BEHIND SOA GOVERNANCE While SOA governance is not a shrink-wrapped capability that you can implement off the shelf without also addressing organizational and procedural issues, at its foundation is the ability to enforce and automate policies across the service lifecycle. This creates the opportunity for software mechanisms that enable the automation and enforcement of governance policies. This section of the white paper outlines the key moving parts of an SOA governance system which performs such policy-related functions.
Stakeholders
Lifecycle Management
Design Time
Policy Configuration
Change Time
Registry Repository
Governance Rules Engine
Policies
Policy Enforcement Points
Run Time
Registry Repository
Message Transport
Management System
Key Components of an SOA Governance System
At a basic level, an SOA governance system should facilitate service-level governance across the lifecycle from design-time to run-time to change-time. It should allow polices to be defined and created, and provide mechanisms for these policies to be enforced at each phase of the service lifecycle. The main components of this system include:
A registry, which acts as a central catalog of business services
A repository, for storing policies and other metadata related to the governance of the services
Policy enforcement points, which are the agents that enact the actual policy enforcement and control at design-time, run-time, and change-time
A rules engine for managing the declaration of policies and rules and automating their enforcement
An environment for configuring and defining policies and for managing governance workflows across the service lifecycle
©2006 webMethods, Inc. All rights reserved.
Page 10
Registry A registry is usually identified as one of the first requirements of SOA adoption and registries play an important role in governance. In simple terms, a registry is a catalog or index that acts as the “system of record” for the services within an SOA. A registry is not designed to store the services themselves; rather, it indicates their location by reference. Having a centralized catalog of services is significant from an organizational perspective because it enables the easy discovery, reuse, and management of services. An SOA registry typically fulfills the following functions:
Stores service descriptions, information about their end-points (the network resource where the service functionality is implemented), and other technical details that a consumer requires in order to invoke the service, such as protocol bindings and message formats
Allows services to be categorized and organized
Allows users to publish new services into the registry and to browse and search for existing services
Maintains service history, allowing users to see when a service was published or changed
As the place where services are made known within the SOA, a registry is also a natural management and governance point. For example, compliance requirements—such as conformance with the WS-I Basic Profile or the use of specific namespaces and schemas—might be imposed on services before they are allowed to be published in the registry. Or, as services are registered or changed, the registry also has the ability to trigger approval and change notification workflows so that stakeholders are alerted to changes. As such, a robust registry is an important component of any SOA governance solution. Another important factor is the interoperability of the registry with other components of the SOA infrastructure. OASIS provides a platform-independent standard for registry interoperability known as UDDI (Universal Description, Discovery, and Integration). UDDI defines a Web services-based programming interface that allows different consumer applications, tools, and run-time systems to query the registry, discover services, and interact as required to provide management and governance capabilities. While it is not a pre-requisite for an SOA registry to be based on UDDI, it is the most commonly adopted standard and ensures the greatest degree of compatibility with other products in the environment.
Repository While the registry plays a central role in policy enforcement, the registry itself does not provide sufficient context for the breadth of SOA governance requirements described in this white paper. For example, policies—in the form of rules and restrictions that are enforced on services—and consumer/provider service level agreements are generally not constructs that are stored in a registry (for one reason, they are not constructs that are
©2006 webMethods, Inc. All rights reserved.
Page 11
known to UDDI). Thus another data store, usually referred to as a repository, is needed for storing governance-related artifacts and supporting the full complexity of managing service metadata throughout the service life cycle. The term “repository” is used in many different contexts—for example, a data repository (SQL database), a document repository (file system), or a source code repository (version control system)—but in the context of SOA governance, the repository can be thought of as a centrally-managed policy store. Among other things, a governance repository should support the following capabilities:
An information model or taxonomy for representing and storing organizational and regulatory policies that can be translated into rules that are enforced by the SOA governance system. It should be possible for policies and rules to be interpreted by people or machines (and sometimes both) as appropriate.
Audit capabilities for tracking the trail of changes and authorizations applied to assets within the repository context.
Identity management capabilities and role-based access controls to ensure that only appropriate parties have access to policies.
A notification system and content validation capabilities to provide additional assurances that policies are well-formed, consistent, and properly applied.
The requirement for a logically centralized repository is particularly important for codifying and enforcing a single “official” set of policies across the organization. However, the actual repository itself may have a federated architecture for scalability and to enable the use of the repository across different geographic regions, multiple lifecycle constituencies, and across corporate boundaries.
Advantages of an Integrated Registry/Repository While the registry and repository have been discussed as separate concerns, in practice there are many benefits to combining the two functions into a single entity. To function properly together as a system, the registry and repository should maintain a consistent view of service definitions, service versions, consumer and user identities, and other information. Implementing them as separate products creates the burden of duplicate data entry, sets up the need to synchronize information, and increases the risk of inconsistencies between the two. When the registry and repository are unified with a single normalized and standardsbased information model and underlying datastore, the integrity of both governancerelated metadata and the information model can be assured. The unified approach also eases the enforcement of policies that apply across the boundary between the registry and the repository. Standard registry capabilities can be offered by an integrated registry/repository with a UDDI interface to the policy repository that allows access to the relevant data.
©2006 webMethods, Inc. All rights reserved.
Page 12
Policy Enforcement Points Thus far, the role of the registry and repository has been established and the case has been made for an integrated registry/repository to create a consistent policy context. These policies, in turn, need to be enforced. Within an SOA governance system, this function is fulfilled by policy enforcement points. The places where policies are actually applied and enforced—the policy enforcement points—change depending on the lifecycle stage. During design-time, the registry/repository itself is the point of enforcement. During run-time, policies are generally enforced by the underlying message transport system that connects service providers with consumers. Finally, during change-time, policies are typically enforced by the IT management system.
Design-Time Enforcement: Registry/Repository Since the registry/repository is the system of record for both service interfaces as well as the assets, attributes, and metadata associated with them, it provides a logical point at which to enforce policies about the design of these particular artifacts. Design-time policies are typically applied as artifacts—which could include WSDL files, schema definitions, process models, and project documentation—are checked into the registry/repository. The following features are desirable in the design-time policy enforcement point:
Identity management — In order to establish rights and responsibility in the registry/repository it is first necessary to identify users (for example, via an LDAPbased identity system or other directory), service consumers, and other participants. Identity is also important for metering usage, logging for audit purposes, and applying approval requirements and other governance processes on an individual or role basis.
Access control — Coupled with identity management, the system should offer fine-grained access configurations over all aspects of registry/repository assets. This includes the ability to secure assets as well as policies, governance processes, and asset classifications.
Automated notifications and approvals — The ability to trigger events in response to Create, Read, Update, or Delete activities in the registry/repository allows alerts, approval processes, content validation scans, and other actions to be automated. These triggers might be applied either before or after the interaction in question. For example, a policy might be established that a design review approval is needed before an object is created in the registry/repository.
Content validation — Content (in the form of assets that checked into the registry/repository) should be scanned and validated according to their type and preconfigured compliance checks. Common validations include WSDL validation, XML schema validation, testing for namespace violations, schema validation, and other interoperability-related scans. For example, service consumers expect interfaces to be well-formed and interoperable, so the registry/repository should
©2006 webMethods, Inc. All rights reserved.
Page 13
automate the process of scanning and assuring that WSDL documents are wellformed and conformable with WS-I interoperability profiles.
Audit trails — A fundamental capability for establishing accountability is the ability to track interactions between participants and the registry/repository, along with the sequence and details of those activities. This record can be used for governance enforcement “after the fact” and to establish usage patterns for guiding process improvements.
Run-Time Enforcement: Message Transport Run-time policy enforcement relies on an SOA infrastructure that is able to exercise policy enforcement in a way that is transparent to, and independent of, the service providers and consumers. This is achieved through an agent or intermediary that resides between provider and consumer and a registry/repository that addresses both the needs of runtime service discovery as well as policy enforcement. The intermediary interacts with the registry/repository to find services and their run-time policies and enforces the policies during the execution of the service. In an SOA, the run-time system is typically a message transport or mediation layer such as an Enterprise Service Bus (ESB). The message transport brokers transactions between service provider and service consumer and frequently offers additional functions such as data transformation, message queuing, reliable messaging, and other operational capabilities.
Consumer
Message Transport Policies
Provider
Registry/ Repository
Web Service
Run-Time Policy Enforcement via a Message Transport Intermediary
Without SOA, the ability to control and manage applications in this manner is restricted both by the scope and the capabilities of the underlying platform. When different applications are integrated, it is generally infeasible to apply a common policy context to the integrated result. A typical challenge is enforcing access security when two applications with different user communities are integrated. For example, application A automatically ©2006 webMethods, Inc. All rights reserved.
Page 14
draws data from application B, but application A users are not authorized to use application B. How do you now control the data that users have gained access to within application A? With the intermediation provided by the message transport, it becomes possible for a distributed network of services to share a common policy-managed context. This is a powerful capability, which emerges as a direct result of SOA. Since run-time policies are typically applied to messages that flow across the message transport system, the types—and level of sophistication—of run-time policies that can be defined and enforced depend on the capabilities of the underlying intermediary. Desirable areas of policy configuration include the following:
Consumer identification and security — Identifying consumer applications to prevent unauthorized access to services; configuring the security of services at runtime and enforcing policies such as encryption (for message confidentiality), digital signatures (for message integrity and non-repudiation), and logging for tracing and tracking.
Routing rules — Configuring run-time routing rules to address performance, version management, and other operational requirements. Variations include: content-based routing (examining a message’s content to determine where to route it according to specific rules, for example, processing certain types of orders via a different process); version-based routing (to support version management and the deprecation of services); and preferential quality-of-service routing (giving consuming applications different processing priorities based on business priorities and defined service levels).
Transformation rules — Translating between different message transports and technology protocols to facilitate application connectivity, or transforming data between consumer and provider.
Service Level Agreement (SLA) management — Policies for managing performance and availability to match the requirements of an SLA, for example, routing a request to a backup service in the event of a failure of the primary service provider, or balancing the request load across additional back-end service to improve performance.
Logging, monitoring, and alerting — Collecting service-level data and establishing rules based on aggregate counters for response time, throughput, errors, and other transaction data so that alerts can be generated when there are violations to predefined SLAs.
Finally, while the intermediary and registry/repository are logically decoupled, a dependency exists to the extent that the intermediary has to understand and interpret the policies defined in the registry/repository. As such, it is advantageous to have a message transport system and registry/repository that are interoperable out of the box; otherwise, this is an integration issue that the implementer has to address.
©2006 webMethods, Inc. All rights reserved.
Page 15
Change-Time Enforcement: IT Management System Whereas design- and run-time policy have “hard” gates – approval for access to the registry/repository and connectivity via the message transport – change-time enforcement relies to a greater extent on IT change management practices and procedures than on enforced control points. Nevertheless, the IT management system—collectively, the set of systems management software tools and services that the IT organizations uses to manage, monitor, and control their business applications and software infrastructure— and the registry/repository combine to play an important role in change-time governance. Unlike previous software paradigms where an application package enters a support or maintenance phase once put into production, SOA involves a dynamic network of interdependent services that are in an ongoing state of adaptation and optimization. Since services, transactions, and SOA events of interest can be monitored by the IT management system, it is a logical source of run-time information that can be fed back into the registry/repository to facilitate the orderly evolution of the SOA environment. This information might include:
SLA-related metrics, such as the average response time, availability, or throughput of a specific service
Process-related metrics in the form of Key Performance Indicators (KPIs), which associate services with user-defined business process metrics (e.g., average order amount)
Business activity monitoring (BAM), alerting, and notification events related to business-level exceptions.
Information such as this can be used to optimize service delivery during the change-time cycle by guiding adjustments in policies, service levels, or in the services themselves. Changes to services will require the change-time governance practices described earlier to be put into effect, for example, performing an impact analysis to assess the implications of changing a service and dealing with the resulting version management issues. As with integration between the message transport and the registry/repository, it is beneficial to have out-the-box linkages between the registry/repository and the management system so that data flows seamlessly between the two without the need for additional integration.
Governance Rules Engine A rules engine is not strictly a requirement of an SOA governance system, but incorporating rules engine technology within the registry/repository enables a significant degree of flexibility and automation, while reducing the reliance on humans to perform mechanical governance tasks (and the associated risk of error). Rules are typically associated with events, while the rules engine handles the firing and chaining of rules. For example, a company’s policy might be that access to services in production is strictly limited to staff belonging to an IT operations role, whereas in the test
©2006 webMethods, Inc. All rights reserved.
Page 16
environment, access is also granted to developers. The rules engine could automate the process of setting and resetting access control switches at lifecycle milestones such as when a service is promoted from development into testing or production. A rules engine also provides the basis for creating complex policies based on reusable templates. In addition to automating governance tasks, the rules engine can also help to deal with policy federation, or the ability to allow multiple policy authors and authorities. This is an important use case for enterprise SOA adoption where governance policies might not be authored and controlled by a single department or organization. A more robust model— which is the basis for policy federation—is to enable both centralized as well as distributed policy creation. Policy federation requires the establishment of guidelines and rules for reconciling policies that come into conflict, and the rules engine assists in the execution of these rules.
Lifecycle Management The final key ingredient of an SOA governance system is the user environment that presents the human interface to the registry/repository and which incorporates the governance lifecycle processes and workflows. Typically, the process workflow includes the following steps:
Publishing of a service by an authorized provider
Discovery of a service by a potential consumer
Requesting use of the service by the consumer
Agreeing on the terms of delivery of the service
Authorizing the consumer
Provisioning of the service
Monitoring of the service delivery
Related to each of these steps, organizations might define approval and notification workflows, exception alerts, and a variety of other process steps. The SOA governance lifecycle management capability will manifest itself in the forms of screens—with information customized to the user’s role within the governance framework—reports, and integration with e-mail to communicate notifications and approval requests. An important feature when services are extended to business partners is the ability to make these lifecycle management capabilities available securely across the firewall. To summarize, an effective SOA governance system should take a comprehensive approach that spans the service lifecycle from design-time to run-time to change-time. It should address the needs of the different roles involved in an SOA and enforce the governance workflows between these roles. Finally, the design of the governance system itself should build on the value and benefits of the loosely-coupled nature of SOA.
©2006 webMethods, Inc. All rights reserved.
Page 17
GETTING STARTED WITH GOVERNANCE Ultimately, SOA governance is about maintaining control over, and having visibility into, the SOA environment in order to engender the level of trust required for ongoing adoption and success. Since many companies are just getting started with SOA, a question that often comes to mind is how to get started and at what point of SOA adoption should governance become a consideration? While effective governance rests in how it is institutionalized within the organization, the key to getting started is to take a strategic approach and to address the “big picture” governance requirements first. Specifically: 1) Make an SOA governance strategy a “must have” component of your broader SOA strategy. Ensure that governance capability-related milestones are synchronized with SOA adoption milestones so that you do not end up trying to retro-fit governance after the fact. Ideally, the right time for governance is before you put any services into place so that any SOA pilot proves out not only the approach itself, but also the related governance practices along the way. 2) Establish a governance roadmap. As part of the SOA governance strategy, there should be a roadmap that defines which specific governance capabilities your organization needs or wants to put into place, and how they will be implemented. 3) Start at the beginning. Generally, organizations will first want to pay attention to the SOA architecture and to design-time governance policies in order to get the SOA journey off on the right foot. In fact, since architecture and governance are what separate a collection of Web services from being a true SOA, organizations that have gone down the path of simply developing and exposing Web services without the appropriate controls or a broader architecture will want to consider an investment in governance. 4) Put a governance system into place. While governance is not a solution that comes in a box, having the right technology framework makes it easier—and in some situations is the only scalable way—to enforce policies and controls. As explained in this white paper, this framework should include mechanisms for defining policies and service contracts and enforcing them through the service lifecycle through workflows, intermediaries, and other automated means. By establishing the right balance between strategy, organizational practices, and supporting technologies, companies will be able to turn the concept of SOA governance into a practical reality and build a lasting foundation for SOA success.
©2006 webMethods, Inc. All rights reserved.
Page 18
APPENDIX: SOA GOVERNANCE REQUIREMENTS CHECKLIST The following checklist identifies key technical and functional requirements of an SOA governance solution and can be used to assess the completeness of your governance implementation strategy.
Registry/Repository Service metadata typing and validation Service relationship and dependency management Search by name, description, keywords, attributes Advanced search (multi-keys, Boolean expressions) Storage of business-level metadata Organization and business unit management Metadata filtering Delegated administration Consolidated auditing on registry/repository lifecycle actions File-system and database persistence options
Service Access Workflow-driven approval/notification request processes User-configurable policies and assertions Service request reporting Consumer data collection
Service Publishing Workflow-driven approval/notification processes User-configurable policies and assertions WSDL validation and conformance reporting Customizable user-defined validation policies (e.g. corporate policies) Command-line publishing utilities Publication wizards
Service Delivery Consumer/provider pair binding Service delivery contract model Contract enforcement, versioning, deployment, monitoring, reporting SLA management Security terms management Request/response routing management Failover/load-balancing management Logging and monitoring management Policy Deployment Management Runtime metrics warehousing
Service Change Management Service subscription management Service metadata subscription ©2006 webMethods, Inc. All rights reserved.
Page 19
Email and SOAP notifications Synchronous/asynchronous notifications
Federation/Replication Master/slave replication Selective synchronization of multiple repositories Selective promotion of objects across repositories
Management Features Advanced policies and account management User activity auditing Taxonomy management API User Management API Access Control API Subscription API Administration API JAXR API
Security Features Fixed and flexible roles Role-based privileges Service | Taxonomy | Attribute | Repository Object Access by Role Access Control Lists (ACLs) Digital signature support LDAP support
Standards Interoperability UDDI v2 and v3 Handling of any URI data types JSR 93: Java API for XML Registries JSR 223: Scripting for Java JSR 94: Java Rule Engine API
©2006 webMethods, Inc. All rights reserved.
Page 20
ABOUT WEBMETHODS, INC. webMethods provides total business integration to the world’s largest corporations and government agencies. webMethods’ flagship product suite, webMethods Fabric, is the only integrated platform to deliver both SOA and BPM, delivering rapid ROI to our 1,400 customers around the globe. With webMethods, customers can take a process-centric approach to their business problems, allowing them to leverage their existing IT assets, dramatically improve business process productivity and ROI, and rapidly create competitive advantage by making their business processes work harder for their company. webMethods (NASDAQ: WEBM) is headquartered in Fairfax, VA, and has offices throughout the United States, Europe, Asia Pacific and Japan.
Copyright © 2006 webMethods, Inc. All rights reserved. Trademarks The webMethods logo, Get There Faster, Smart Services and Smart Processes are trademarks or registered trademarks of webMethods, Inc.
Worldwide Headquarters 3877 Fairfax Ridge Road South Tower Fairfax, VA 22030 USA Tel: 703 460 2500 Fax: 703 460 2599
US West Coast 432 Lakeside Drive Sunnyvale, CA 94088 USA Tel: 408 962 5000 Fax: 408 962 5329
European Headquarters Barons Court 20 The Avenue Egham, Surrey TW20 9AU United Kingdom Tel: 44 0 1784 221700 Fax: 44 0 1784 221701
Other product names used herein may be trademarks or registered trademarks of webMethods or other companies. Statement of Conditions webMethods, INC. PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OR CONDITIONS OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL WEBMETHODS BE LIABLE FOR ANY LOSS OF PROFITS, LOSS OF BUSINESS, LOSS OF USE OR DATA INTERRUPTION OF BUSINESS, OR FOR INDIRECT, SPECIAL, PUNITIVE, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY KIND, EVEN IF WEBMETHODS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES ARISING FROM ANY DEFECT OR ERROR IN THIS PUBLICATION OR IN THE WEBMETHODS SOFTWARE.
Asia-Pacific Headquarters 6 Temasek Boulevard, #24-01 Suntec Tower Four 15 Blue Street Singapore 038986 Tel: +65 6389 3200 Fax: +65 6389 3299
www.webMethods.com
webMethods, Inc. may revise this publication from time to time without notice. Some states or jurisdictions do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you. All rights reserved. No part of this work covered by copyright herein may be reproduced in any form or by any means—graphic, electronic or mechanical— including photocopying, recording, taping, or storage in an information retrieval system, without prior written permission of the copyright owner.
RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the U.S. government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 (October 1988) and FAR 52.227-19 (June 1987).
©2006 webMethods, Inc. All rights reserved. The webMethods logo and Get There Faster are trademarks or registered trademarks of webMethods, Inc. All other names mentioned are trademarks, registered trademarks or service marks of their respective companies.
Related Documents
Soa Governance
November 2019 16
Guide To Soa Governance
November 2019 13
Soa Policy Governance
November 2019 4
Softwareag Soa Governance
November 2019 8
Introduction To Soa Governance
November 2019 21