Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Enterprise SOA Reference Architecture
Tushar Jain http://www. architecture-soa-bpm-eai.blogspot.com
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 1
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Contents Enterprise SOA Reference Architecture........................................................................................................3 Data Persistent Layer.....................................................................................................................................3 Service Providers...........................................................................................................................................4 Services Layer................................................................................................................................................4 Enterprise Service Bus...................................................................................................................................5 Presentation Services.....................................................................................................................................5 Human .......................................................................................................................................................5 Machine .....................................................................................................................................................6 Integration with Outer World.........................................................................................................................6 Security..........................................................................................................................................................6 Governance....................................................................................................................................................7 Rule/Policy Repository & Engine..................................................................................................................9 Service Directory...........................................................................................................................................9 Event Management........................................................................................................................................9 Generic Event Handling.............................................................................................................................9 Time Based Event Handling....................................................................................................................11 Heart Beat Event Handling......................................................................................................................11 Transaction Management.............................................................................................................................12 Exception & Error Management......................................................................................................................15 Relationship.....................................................................................................................................................18
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 2
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Enterprise SOA Reference Architecture
Here I propose SOA Enterprise Reference Architecture. This is for enterprise and certainly for an application it should be inspiration.
Reference Architecture is independent of platforms offered by various vendors. It is also possible that none of the vendors in market provide complete solution to realize this reference architecture. Data Persistent Layer Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction and Meta. It may also include Datamart and Data driven reporting infrastructure.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 3
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com In most of the large enterprises Data Persistent Layer consists of relational (RDBMS), hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and any other type. These variations in data characteristics warrant unique data handling and treatment technologies. Service Providers This Layer consists of different business applications. These applications may be packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies (e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy once, which do not expose their functionalities as service. At this layer EAI exist. Some may argue that SOA is one form of integration then how EAI can exist at this layer. I will discuss this argument in one of my future posting but not now. Services Layer This layer is consists of multiple sub layers which encapsulate different class of services. It encompasses Connectivity Services, Data Services, Business Activity Services and Business Process Services. Each layer in turn consists of other sub-types of services. This layer exposes different aspects of business - functional and infrastructural aspects. Connectivity and Data services are consisting of Infrastructural services while Business Activity and Process services expose functional Services. These services though in layers but still breach layered architecture to expose these services independent of layers and for better performance. These services can also be exposed to multiple ESBs if situation warrants. 1. Connectivity services primarily cater to accessing system level functionalities, any adapter & connector services and partner (B2B) integration services. 2. Data services comprises of services, which caters to Business, Technical and Meta data related services. 3. Business Activity Services consist of Atomic business/functionalities and any custom business logic developed by combing one or more Data & Connectivity services. 4. Business Process Services primarily consist of Process integration, Machine and Human centric services. These services primarily comprise of Business Activity services. Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 4
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Enterprise Service Bus ESB provides four functionalities: Orchestration Routing Mediation Enrichment Some ESBs may provide one more functionality - Choreography. This functionality is associated with BPM systems and currently none of Commercially available ESB provides the same. Enrichment may be of two types: Data Transformation and Channel Transformation. Most of the analysts & architects keep Data Transformation & Channel Transformation separate functionality.
Presentation Services These services support various user interfaces (human or machine centric) and let outer world (intra & inter enterprise) interact with enterprises IT assets. The consumers of presentation services are Human Thin Client
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 5
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com Browser clients: May be based on traditional HTML based or AJAX based Portable devices: May be based on open standards (WAP) or propitiatory (device or carrier specific) Thick Client Alway Connected: These clients connect to service providers and consume the services in real time. Traditional think client falls in this category Offline Clients: These clients do not connect to service providers in Always On fashion but an actor can work in offline mode and when ever connectivity available, services consumed. Java WebStart and Adobe AIR are two most widely used technologies in this field. Voice Based Client Voice based clients are gaining popularly to reduce load on contact center. To facilitate this type of user interface, IVR plays great role. Machine These types of interfaces are not very obvious but very prevent at enterprise level. Application level interaction (Software to software) and interaction with various devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa) are part of this type of interfaces.
Integration with Outer World Most of the large enterprises also get integrated with external world. This integration is both ways - information flow is in both directions. To further complicate the scenario, enterprises increasingly using SAAS. This situation requires special attention and tools. In case of integration with business partners B2B Engine plays major role in conjunction with Partner services. SAAS services may be consumed directly or via B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should be bypassed but for m/c interaction or substantial data transfer (Document management for insurance companies) B2B engine should be engaged. Security
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 6
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Security in SOA is one of the biggest concern due to distributed & heterogeneous systems interacting with each other. The security related aspects should be handled at each level of enterprise. In the proposed model security is considered from physical and virtual perspective. It also takes into account security from network and application view. At monitoring front it take care of Technical - For support staff, Process & Managerial - For business process owners & managers and Executive - for top rung of business leadership requirements.
Governance Governance is one of the burning issues in SOA environment. To discuss SOA Governance's Technical architecture, it is necessary to understand SOA Governance model. Following figure depicts the SOA Governance Model.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 7
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
The picture below intends to depict SOA Governance Technical Architecture.
Service Life Cycle Management Service Life Cycle Management essentially helps in managing service life cycle from its conception to decommission. This is one of the vital components, which determine the success of SOA initiative in any enterprise and also its long-term viability. Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 8
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com Rule/Policy Repository & Engine Rule/Policy Repository contains various Business Rules and Governance & Security Policies. The engine part of the same will help in execution of Rules and Policies as and when demanded. Service Directory Service directory essentially consists of UDDI, which facilitates consolidated listing of services. Event Management To exploit SOA to its full extent event handling plays crucial role. In SOA environment Events can be classified into three categories 1. Generic Event Event fired/raised in response of event or interaction with other service/s. 2. Time Based Event Event fired/raised at particular moment of time to perform certain task 3. Heart Beat Event Event fired/raised by services at regular interval to announce its existence and may be level of throughput it is working. Generic Event Handling These are events which fire/raise other events or results due to interaction of services in ecosystem.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 9
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
1. Event Generator generates events with minimal data set. 2. These events should be formatted as per the event protocol defined in ecosystem, so that theses event are available to entire ecosystem for further processing. Ø Event Pre-processing: Check that mandatory attributes are present in the incoming events Ø Event Refinement: Introduce the other system-generated attributes like start time etc. Ø Situation Refinement: Check the target status on which action would be performed Ø Impact assessment: Required result assessment after that impact applies on the target 3. Event factory determine the specific type of action, which the given event have. 4. Event listener executes the specific action, which stated at the specific action code. There we have some general action apply to all type of system is present in generic action code and specific to specific action is present in the specific action listener 5. Event listener executes the action and calls the required system to complete it.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 10
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com 6. Some event, which completed their actions, and no further support needed from any other services are terminated 7. Some events which need further processing from some other services or may be generate other event in response of event is send back to the event engine 8. Event in response of event is again run as a new event and again it initiates its processing cycle in the event engine Time Based Event Handling Scheduled event delivered to a service at specific time in the future. Event deliveries can be deferred for short periods of time (such as seconds or minutes) or for long stretches of time (for example, hours later for batch processing). Until that delivery time, the Event is essentially invisible until it is delivered, allowing you to schedule work at a particular time in the future.
Heart Beat Event Handling Heartbeats, a common software construct, verify the continual operation of a specific service. With heartbeats, a targeted service continually broadcasts a signal across its environment. System works normally when client services can detect a targeted service's heartbeat signals
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 11
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com There is number of interdependent services in SOA ecosystem such that, if one failed, others will also fail. Heartbeat generally propagated across SOA ecosystem using publish/subscribe paradigm.
Transaction Management Transaction Management is one of the most difficult aspects of any enterprise architecture due to varying level of understanding among stakeholders, various technological platforms involvement and spread of transactions – intra & inter enterprise.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 12
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com In general any enterprise grade transaction framework follows ACID but due to presence of long running business processes and involvement of asynchronous communication traditional ACID becomes irrelevant. To accommodate long running business processes and asynchronous communication relaxed ACID is used which allows violation of ACID but with capability of correcting the violations in reasonable time period and with little business impact.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 13
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 14
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com SOA capable enterprise grade Transaction Framework should have following features: · · · · · · · · ·
Multi level undo/redo Coalescing of transactions Automatic aggregation of nested transactions Supports batching of transactions (Aggregation) Rollback of aggregated transactions for error recovery Listener support Synchronous and Asynchronous Transaction Compensatory Service Transaction Propagation Exception & Error Management
Exception & Error Management is consist of Exception & Error Handling Exception & Error Logging Exception & Error Notification In any enterprise under SOA ecosystem Exception & Error Management poses following challenges: Exception & Error Handling No predefined format for passing exception information among participating services Web services do not have capability of maintaining stack trace Lack of common exception vocabulary across enterprise Exception & Error Logging Scattered logs across services/applications Varying formats of logs across services/applications Toggling logs at run time is service/application dependent Exception & Error Notification Every enterprise/LOB has its own unique notification requirement Toggling Notification at run time is necessity SLA and exceptions are coupled To counter these challenges there must be a comprehensive Exception and Error Management framework which can manage at enterprise level and form the foundation for Remediation framework.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 15
Tushar Jain Enterprise SOA Reference Architecture
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com On the high level, Exception and Error Management framework’s architecture can be depicted as follows:
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 16
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Remediation Framework Remediation framework plays very important role in any SOA ecosystem. It can be though of extension of Error & Exception Management and Transaction Management frameworks. Remediation framework leverages Error & Exception Management Framework by utilizing logged error & exceptions. It also supplement Transaction Management Framework via helping to implement relaxed ACID where human intervention might be needed. Remediation framework also makes whole of SOA ecosystem robust via utilizing compensatory services.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 17
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Relationship In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each face of SOA ecosystem affects other.
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 18
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 19
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture
Tushar Jain
[email protected] http://www. architecture-soa-bpm-eai.blogspot.com
Enterprise SOA Reference Architecture 20