Context
In the digital age, the focus of enterprise/business/application/etc. architects is not the thing (strategy, policy, service, rule, application, process, etc.) – the focus is how the thing changes and how things change together.
In addition to being cheaper, faster, better it is mandatory to become to more agile, more synergetic (i.e. IoT), more comprehensive.
- Digital eats physical: Everything becomes digital: products, information, content, documents, records, processes, money, rights, communications.
- Fast eats slow: As digital is intangible thus news tools and new execution speed immediately.
- Group eats single: It is mandatory to collaborate to address modern complex problems.
- Big eats small: Digital things are at new scale.
Introduction
This blogpost outlines how BPM can enable changes which accelerate improvements and innovations in the digital age.
This blogpost is based on the blogpost “#BPM for software architects – from monolith applications to explicit and executable #coordination of #microservices architecture” http://improving-bpm-systems.blogspot.ch/2014/08/bpm-for-software-architects-from.html (referred as “base” blogpost below).
The goal of IT in the digital age is to be able to provide software-intensive solutions which are easy to evolve instead of classic monolithic applications which are difficult to evolve. This blogpost shows how to design and build process-centric solutions which are easy to evolve. Such solutions are explicit and executable aggregates of components. Aggregates are organised around business processes and components are microservices which wrap various process-related artefacts.
Note: Considering that microservices are autonomous units of functionality, one monolithic application as a big unit of deployment may become a few hundred of microservices as small units of deployment (although the size does not matter in this case).
Both aggregates and components (some of aggregates are components as well) will be analysed from the change (evolution) point of view. In other words, how to carry out changes of each particular artefact and all afterfacts together without breaking the system - enterprise as a systems of processes ( see http://improving-bpm-systems.blogspot.co.uk/2014/03/enterprise-as-system-of-processes.html ) and for achieving the enterprise goals.
Note: Evolution is related to the impact analysis, dependency management and optimisation.
It is considered that all artefacts are versionable and several versions of the same artefact may co-exist in the company’s computing environment. Traceability considerations are at maximum – everything (including changes and work done) is logged as records.
All artefacts are wrapped as services (actually, microservices). A process coordinates (with the use of various coordination techniques – see http://improving-bpm-systems.blogspot.co.uk/2014/03/coordination-techniques-in-bpm.html ) various services and it is a service itself. Thus process is an explicit and executable way to aggregate smaller services into bigger ones. In other words, a process is an aggregated service.
Versioning of artefacts (10.3 from by book http://www.samarin.biz/book )
To achieve the versioning of artefacts it is necessary to understand how to treat relationships between artefacts (see 2.4.4 of the book).
We recommend that a system be evolved via some kind of transformation cycle as shown in Figure 1. Start with a stable configuration of approved artefacts. Then introduce a new version of the artefact B3 which is available only for one consumer (i.e. artefact A2) which has to be also versioned. After achieving higher confidence with these new versions, switch all other consumers (i.e. artefact A1) to the new version of the artefact B3. When it is considered that all new artefacts are functioning correctly, their old versions can be removed. The transformation is over and a stable configuration of approved artefacts is once again reached.
Figure 1 Figure 10.3 “Transformation cycle” from the book
In a properly architected system, you may carry out several transformation cycles at the same time.
Process-template and process-instance
A process-centric solution have several processes (actually process-template - a formal description of the process) and some stand-alone services (e.g. a stand-alone service may generate an event which launches one of the processes (actually the process-instance of a process-template - enactment of the process template).
The distinction between process template and process instance is very important. The life-cycle of process-template is controlled at design-time. The life-cycle of process instance is controlled at run-time. A process instance is created, maybe suspended & resumed and finally terminated. Many process instances may co-exist at the same time as shown in Figure 2.
Figure 2 Templates and instances
Process-centric artefacts
Process-centric artefacts and relationships between them are the following:
- The business is driven by events
- For each event there is a process to be executed
- Process coordinates execution of activities (automated and human and sub-processes)
- The execution is carried out in accordance with business rules
- Each activity operates with some business objects (data structures and documents)
- A group of staff member (business role) is responsible for the execution of each human activity
- The execution of business processes produces audit trails
- Audit trails (which are very detailed) are also used for the calculation of Key Performance Indicators (KPIs)
Events
Evolution of an event is very straightforward – just new version for any change. Usually, there is a mapping (or decision) table (implemented as a “dispatch” service – see 2.6 of the base blogpost) to provide the correspondence between events and processes. In the simplest policy, a particular event is linked to a particular process template (or a particular version of a particular process template). More sophisticated policies are possible, e.g. usage of the most recent version, time-based selected, etc.
Note: Events mays be generated by processes.
Note: Events may be processed via EPN and decision management techniques.
Potential side-effects (evolving together): None at the moment (just explicitly ignore event if it does not launch any processes).
Process-template
Evolution of a process template is evolution of a composite object. The simplest policy is a very strict binding (also called “early binding”) – a particular version of the process template refers to a particular version of each components (actually microservices).
Figure 3 Early binding
More sophisticated policies are possible, e.g. process-template is using the most recent versions of each component available at the process-instance launching moment (also called “late binding”). Because, the process-template is actually a description then its versioning is not a big problem.
Potential side-effects (evolving together): As life-cycles of a particular process-template and its process-instances do not match, it is necessary to understand what should be done with the running process-instance in case of changing the process-template, although the process-template and its process-instances are different objects (similar to mother and her borne children).
Process-instance
Process-instance is a composite object and its evolution is better to avoid (like changing a running car). Evolution of a process-instance may be necessary for some legal purposes if a long-running process-instance should be modified in accordance with the evolution of related process-template. The related technique is described in http://improving-bpm-systems.blogspot.ch/2010/03/practical-process-patterns-mint.html . Of course, it is better to avoid evolution of process-instance at all, but small changes should be possible.
In practice, the main reason to evolve process-instance is for correcting various errors and exceptions, e.g. in data or in rules or in automation. If some of the components are expected to be quickly evolving or “shaky” then the relationships between the composite and these component should be indirect thus manageable externally.
Figure 4 indirect binding
Sometimes, it is necessary to create a version of an external component which must be used only by a particular process-instance. In general, all external components are re-usable from various aggregates.
Roles
Roles should be define in a suitable DSL externally from the process template and changes for a particular process instance should be possible. Usual technique is to have a set of dedicated functional roles (Responsible, Accountable, Consulted, Informed) for each human activity within a process and be able to provision these roles by various organisational and other roles externally from the process template.
Rules
Rules is a typical service which is implemented in a DSL (decision management notation). This service is stateless and easy to evolve.
Audit trails
Audit trails are easy to evolve. Ii is important to define them explicitly in processes, for example, like measurement-points. Audit trails must be kept outside the process engine in, for example, an enterprise data warehouse thus be independent from evolution of BPM suite itself. Typical process execution data (start/finish time for each activity, etc.) must be merged with some business data to associate separate process-instances which treated the same business objects.
KPIs
If audit trails are done correctly then KPIs are easy to evolve.
Human activity
Human activity is implemented as an interactive service. Sometimes, such a service is a generic tool (which is external to process-template) and such a tool should receive from the process-instance a reference to a human activity to be treated. This is an example of indirect relationship mentioned above.
Sub-processes
Typically, early or late binding is applied for selecting a version of a sub-process to be used (although it depends on capabilities of the business process engine). In the majority of situations, the late binding works fine – just remember to record the version of a sub-process template used in each invocation.
Data structures
As a good practice, business data structures are kept in a generic format (e.g. SDO) and transferred along the process as a black box. To implement some routing logic, additional technical or process-template-specific data structure is created. Bridging between business and technical data structures is done by in automated activities.
Documents
Documents are kept in external repositories, e.g. a document management system or ECM tool. They are referred via URLs and some metadata.
Automated activities
Automates activity is the most “shaky” component of the process (as an aggregate). The indirection binding which is used for them is done through a “robot” (see 2.3 in the base blogpost). Robot is a very stable service and the process-instance passes to it the name of the automation script to be executed as well as input and output parameters. The name of the automation script is a process parameter (thus changeable by the process-template administrator and the process-instance administrator) and input/output parameters are SDOs.
The typical error recovery practice discussed below. Figure 5 shows a “container” in which an automated activity “A” operates within the processes. The normal execution sequence is “E1-A-E2”. Because the automated activity may fail, the container contains the intermediate exception event "E3" and an activity for Error Recovery Procedure (ERP).
Figure 5 Error recovery loop and Error Recovery Procedure (ERP) – exception handling
In case of failure, the recovery execution sequence will be “E1-E3-ERP-E1-A-A2”. ERP may be very trivial (just try again) or more intellectual (try three times and then ask a person to have a look at it).
In additional to exception, it is necessary to define time-out to prevent endless automated activities as shown in Figure 6.
Figure 6 Error recovery loop and Error Recovery Procedure (ERP) – exception and time-out handling
Automation activity is an automation script which is executed by robot. Typical automation script is an aggregate (usually in an interpreted language) of several micro-services and this aggregate should be executed as a one transaction (see Figure 7).
Figure 7 Execution of an automation script by the robot
Again, normal execution sequence is “E1-A1-A2-A3-E2”. In case of failure of “A2”, the sequence will be “E1-A1-A2-E3-ERP1-E1-A1-A2-A3-E2”. The double execution of “A1” is possible because of that all micro-services are idempotent (see 2.10 in the base blogpost). If “ERP1” is a human activity then the correction of “A” automation script may be carried out within this human activity.
Note: Processes with only automated activities must be idempotent.
Of course, there is no a robot per each automated activity because the robot must be able to handle concurrently several automation scripts as the same time (as several process-instances of the same process-template may be executed at the same time). Instead, there is a queue of jobs for a group of similar robots. An automation activity of a process-instance puts an automation script into a queue and waits for a robot to execute this script and inform the process-instance that this automation activity is completed (see Figure 8).
Figure 8 Queuing of jobs for robots
The queue is shared between various process-instances and it is possible to have several specialised queues. The queue size and robots are monitored.
In some sense, robots work as humans – wait for a job from process-instances, execute jobs when they can and inform a particular process-instance that a particular job is completed.
Conclusion
The describe approach was used (since the year 2000) for a production system comprising about 3 000 complex products per year, 50 persons, about 50 different activities, 3 production chains, 6 repositories and 40 IT services (actually, a couple hundred of micro-services). The system was in place for several years. The maintenance and evolution of this production system required several times less resources. Also, several successful (and easy to do) migrations of its big components were undertaken.
Thanks,
AS
Posts
No comments:
Post a Comment