Post-platform enterprise pattern: EEYORE

Post-platform enterprise pattern: Enterprise-Enterprise Yet Open Robotic Environment (EEYORE)

With thanks to John Morris (@JohnHMorris) and Michael Poulin (@m3poulin) for their valuable comments.

An updated version of the article is available at https://bpm.com/blogs/post-platform-enterprise-pattern-faster-and-cheaper-inter-enterprise-ecosystem-business

WHY

In the modern business world, the vast majority of work is done by joint efforts of two or many enterprises. Inter-enterprise “working together” is a norm right now. It may have different longevity: one-off or sporadic or permanent (as B2B partnership). Can such “working together” be used systemically? For example, a modern enterprise is trying to get competitive advantage by

1. perfecting, digitizing and innovating its core-business capabilities, and 
2. using the best “other” enterprises for provisioning “other” capabilities needed to achieve the particular goal (even its mission); note that, those “other” capabilities for the enterprise point of view are the core-business capabilities from “other” enterprises point of view. 

In other words, an enterprise may combine its own “internal” capabilities with “external” capabilities obtained from other enterprises to achieve the particular goal.

Thus every enterprise involved is carrying out only its core-business capabilities! This is a big difference with the current situation in which some of enterprise’s capabilities (i.e. core-business) must compete in the world-wide market and some of enterprise’s capabilities (supporting) have no competitor at all.

The economist Ronald Coase - said that “Firms exist when the transaction cost of doing something within the firm, even with all its overhead, is lower than cost of doing things through a marketplace of free agents”. At present, an enterprise possesses core-business capabilities and supporting capabilities. Historically, the cost of intra-enterprise transactions is lower than the cost of inter-enterprise transactions. At the digital age the costs of inter-enterprise transactions are constantly dropping as digital technology develops (the so-called "API economy" is a good example of this). Of course, the cost contracting out versus direct management is only one of many factors such as risks, time lag, security, etc. to be taken into account.

Thus, the famous classic representation of an enterprise may be redrawn as shown in the illustration below. Note: Also logistics, marketing and sales may be considered as non core-business capabilities and be provided by “other” enterprises.

Ability of several enterprises to work together with maximum synergy and minimum overhead (thus much better than a classic enterprise) is critically important. Potential list of common activities to achieve that is not long but daunting:

• formally define a work to be done ; 
• find the best other enterprises for this work to be done; 
• contract selected other enterprises; 
• activate and configure a trusted working environment for all participating enterprises; 
• carry out the work with secured sharing some data and information among participating enterprises and 
• complete all contracts related to this work to be one. 

Certainly if modern digital technologies are properly architected together then they can enable this new way of working by making it more efficient and more effective that the current way of working.

Explicit, formal, machine-readable and machine-executable processes can act as a neutral and natural referee for coordinating inter-enterprise work. A potential implementation is an BPM-suite tool. A target is to position a BPM-suite tool as an independent and trusted 3rd party (a referee or a coordinator) to execute legally-bound processes for 2 or more mutually non-trusted parties. See http://improving-bpm-systems.blogspot.com/2016/07/digital-contract-as-process-enables.html

Of course, when many enterprises contribute into some common work, records management must be centralized (i.e. accepted by all the participants) and suitable for digital way of working. A digital archive with good availability and integrity can act as an escrow for some transactions and documents. Potential implementation is the blockchain technology.

An example of such a “disaggregation” of a classic enterprise – https://news.cgtn.com/news/35636a4e33637a6333566d54/share_p.html

HOW

Working together for the same goal requires sharing some things (business and technical artefacts) and, probably, some agreed means. Such a sharing is lesser for cooperation (different enterprise for different activities) and greater for collaboration (different enterprises for same activities).

Imagine that 10 enterprises have decided to work together. Shall each of them implement business processes required for this engagement? Add complex EDI infrastructure for communicating between processes from different enterprise? Keep everything within each enterprise? It looks a bit difficult. If something that must be shared is done once for everyone then everyone will gain. Thus some processes related to this engagement may be implemented once at an independent provider and linked to existing processes in each enterprise.

 

WHAT to share	Examples	Reason(s)	Supporting means
Data (see the DIKW pattern)	hashes, cryptographic keys	Security	common immutable storage
Information (see the DIKW pattern)	business objects	Interoperability, e.g. in interfaces	common immutable storage
Facts	audit trails, signed documents, agreed goals, demonstrated KPIs, inputs & outputs of inter-enterprise transaction	Traceability	common immutable storage
Events	some facts important for business	synchronization of work	common event queue
Rules	agreed business logic	regulation of work	common decision mechanism
Activities	agreed procedures, agreed SLAs, agreed protocols	automation	common scripts in domain-specific languages
Processes	agreed testing methods	coordination of work	common coordination mechanism
KPI calculations	agreed formulas and results of calculations	common view on performance	common dashboard
Intelligence	agreed algorithms and results	common view on planning	common prediction mechanism
Improvements	some proposals for better working	better joint performance	common set of digital models and agreed scenarios
Estimations of uncertainty which are evolving along the time	participants’ opinions on some factors	non-biased corporate security for all the enterprises involved	common immutable storage
Estimations of risk which are evolving along processes	participants’ opinions on adverse impacts	non-biased corporate security for all the enterprises involved	common immutable storage
Finances	payment mechanism	minimizing transactions with banks	own “local” currency

Things, which are shared in the particular case, must be a) architected together and b) digital (explicit, formal, machine-readable and machine executable) to achieve desired values of the important emergent characteristics such as:

• the transactional cost for inter-enterprise transactions, 
• risks, 
• time lag, 
• security, 
• manageability, 
• etc. 

WHAT

New application architecture for inter-enterprise agile engagements can be built on the following pillars:

• Immutable common digital archive (FACTS) to store all versions of all the artefacts. 
• Microservices (ACTIONS), potentially in a serverless computing environment (because a digital archive is used as a common immutable storage). Please note that microservices are normal services (according to OASIS) except that one unit-of-functionality is one-unit-of-deployment is one-unit-of-execution. 
• Machine-executable processes (COORDINATION). 

These pillars perfectly work together (in a robotic way without high level of creativity) – process templates define the granularity and interfaces for microservices and digital archive keeps all the artifacts (of course, they must be versioned). Process instances define validity of various facts.
See also http://improving-bpm-systems.blogspot.com/2018/06/architecting-modern-digital-systems.html

Some hurdles – processes modelling

Modelling of inter-enterprise processes is not easy because such processes are distributed (each of them may be executed in its own computing environment) and they communicated by exchanging of messages thus it is necessary to handle exceptions in communicating processes. Good modelling styles and process patterns will help.

Some hurdles – process execution

Some BPM-suite tools are already connected to the blockchain which is used as a common immutable storage.

1) Ultimus can use blockchain as a data storage - "Ultimus uses Tierion to create an audit trail for business processes and prove the integrity and timestamp of documents." See https://medium.com/tierion/ultimus-integrates-tierion-for-blockchain-digital-process-automation-f8331d76216a

2) Bonita can use blockchain – https://www.youtube.com/watch?v=lkwvko2Uy24

3) ConsenSys uses Camunda BPM-suite to use blockchain as a data storage, creating new users and executing smart-contracts https://www.youtube.com/watch?v=oww8zMzxvZA&feature=youtu.be
Some hurdles – process execution as a smart contract

One of the option to bring process to existing blockchain implementations is to implement a BPMN-like interpreter as a smart contract in the blockchain technology.

Some hurdles – blockchain mentality

People from the blockchain domain follow the “if you have a hammer then everything is a nail” negative pattern. https://blog.apla.io/report-of-the-egaas-team-556e5e4717bd

See http://ipe-lab.com/publication/223/?sphrase_id=41 as a case of the blockchain for control of third-party services. This an attempt to change a group of processes (from a few B2B partners) into a blockchain application. On the bottom picture (TO-BE), actual processes from the top picture (AS-IS) were lost. And, “Erroneous behaviour is judged by voting among all participants”, not by thorough analysis!

Conclusion

This pattern opens new perspective for the traditional Business Process Management which becomes also Inter-Enterprise Ecosystem Business. In other words, processes will expand outside enterprises to enable faster and cheaper “working together” for enterprises. 

And a quote from Alberto Manuel‏ (@AlbertoManuel) "Architecting expanded Value chain in the era of digital ecosystems and shared capabilities."

Thanks,
AS

Architecting modern digital systems #entarch #bizarch #apparch #bpm #security #microservice

Mini-course at VFU

1 Title

Architecting modern digital systems

2 The problem to be addressed

At present, there are many IT-related methodologies, technologies, tools and schools of thoughts which overlap and contradict each other. The best practices are actually the best only in particular situations. Often decisions about software-intensive solutions are taken on the in-complete and subjective base. All of this tremendously complicates the modern digital systems thus reducing their potential effectiveness and efficiency. 

3 Objectives

The purpose of this course is to provide the basic knowledge and experience necessary to better understand how to deal with the increasing complexity of the information technologies to obtain the synergy between business needs and IT potentials. 

4 The approach

The course is based on the practical use of Enterprise Architecture (EA) which a methodology and practice for architecting solutions. EA provides an overarching guideline for understanding a “problem space” and take necessary decision about the “solution space” to deliver which addresses the problem. 

5 Learning outcomes

The trainees will

• learn a systematic approach for architecting digital solutions; 
• learn about some modern information technologies; 
• learn how those technologies are working together for a systematic architecting, design, implementation, operations and evolution of digital systems; 
• carry out a practical architecting exercise. 

6 Target audience

Bachelor and master level students specialising in IT. 

7 Requested knowledge

General knowledge of IS/IT. General programming experience. 

8 Layout of the teaching

Teaching will be given as six 1.5-hour lectures. The first 4 lectures will be about presenting some methodologies and technologies. Then the students will be asked to architect a solution based on a practical situation. The last session will be about presenting the student’s solution and wrapping up this mini-course. 

9 Agenda

• Introduction https://www.slideshare.net/samarin/minicourse-at-vfu-architecting-modern-digital-systems-0 
• Architecting digital systems https://www.slideshare.net/samarin/minicourse-at-vfu-architecting-modern-digital-systems-1 
• Business Process Management https://www.slideshare.net/samarin/minicourse-at-vfu-architecting-modern-digital-systems-2 
• Security https://www.slideshare.net/samarin/minicourse-at-vfu-architecting-modern-digital-systems-3
• Microservices https://www.slideshare.net/samarin/minicourse-at-vfu-architecting-modern-digital-systems-4
• Practical work – architecting a solution https://www.slideshare.net/samarin/minicourse-at-vfu-architecting-modern-digital-systems-5
• Review of the practical work 

Thanks,
AS

Smart Cities from the systems point of view

Thanks,
AS

#GDPR as an #BPM application

This blogpost explains how to implement the EU General Data Protection Regulation (GDPR) by design and by default via Business Process Management (BPM). This blogpost describes only a reference solution architecture without much implementation details. It focuses, primarily, on the artefacts such as capabilities, rules, roles, data-structures, documents, explicit coordination and audit trails.

1 Terminology in the GDPR

Although the information security domain is developed well, the GDPR document (see article 3) uses rather exotic terminology (sources are not provided). For example, many concepts, available from the standard privacy framework (ISO/IEC 29100) have different designations (terms). Another example is that the concepts “data” and “information” do not follow the DIKW “pyramid”.

Although a mapping between 16 terms of the GDPR and existing terminology is not difficult, it would be better to avoid such a mapping.

2 The main element

The main element of the GDPR is a data-structure object “Personally Identifiable Information” (PII) in the ISO/IEC 29100 terminology or “personal data” in the GDPR terminology. It must be explicitly and carefully protected:

• for its confidentiality, integrity and availability
• in rest, in transit and in use
• throughout its life cycle

Usually, the life cycle of PII objects is very simple and covered by 4 actions which are known as the CRUD pattern (although each update may create a new version of the PII object). 

3 The core processes

Those actions, namely, Create, Read, Update and Delete, must be presented as small processes (or workflows) to provide the design and execution traceability. Considering that any PII is owned by the “PII principal” (or a natural person to whom a set of personally identifiable information relates), he/she must approve some actions on his/her PII object.

For example, an PII principal must provide his/her consent to process his/her PII object and such a consent must be kept as a record by the “PII processor” (privacy stakeholder that processes personally identifiable information on behalf of and in accordance with the instructions of a “PII controller”).

4 Some supporting capabilities

The core (or life cycle) processes use several capabilities (services or processes) such as:

• identity management
• access management
• anonymization 
• encryption
• etc.

Also, some error- and exception-handling processes are necessary to properly handle privacy incidents.

5 Related roles

The essential roles are the following:

• “PII principal” in the ISO/IEC 29100 terminology or “data subject” in the GDPR terminology – the owner of the PII, 
• “PII processor” – persons or organisations who/which execute the GDPR processes,
• “PII controller” – authority which, alone or jointly with others, determines the purposes and means of the processing of personal data, and
• “Data Protection Officer (DPO)” – person who is the owner of the GDPR processes.

6 Rules

The execution of the GDPR processes is guided by numerous rules which are; actually, the majority of the GDPR documents. For example, if a PPI principal is a citizen of an EU countries then “PII processes” must follow the GDPR.

Unfortunately, it is unknown if these rules comply to the MECE principle (no overlaps and no holes). 

7 Complex scenarios

There are a few scenarios which involve more than one PII object. For example, split, merge, export, transportation, correlation, etc.

8 Conclusion

Use of BPM to implement the GDPR addresses all the GDPR concerns. Explicit and machine-executable processes are mandatory to achieve by design and by default the listed below key points.

• Compliance – all privacy-related activities and coordination between them can be easy analysed.
• Accountability – generated audit trails provide factual and objective information about why did what.
• Data protection officers (DPOs) – as a role which owns all the GDPR processes.
• Consent – is achieved by the design of the GDPR processes and records management.
• Enhanced rights for individuals – is achieved by the design of the GDPR processes.
• Privacy policies – all PII controllers and PII processors must analyse their privacy policies via the logic of explicit processes.
• International transfers – also become processes.
• Breach notification – as an integral part of the privacy incident GDPR processes.

Thanks,
AS

Domesticate the #IoT as cyber-physical systems

This article is a continuation of “#IoT as a system of digital contracts” article ( see http://improving-bpm-systems.blogspot.ch/2016/08/iot-as-system-of-digital-contracts.html ).

1 Introduction

The recent IoT-based DDOS attack confirmed the urgent necessity for more serious and systemic integration of the IoT into our civilisation. At present, many devices from the IoT “world” act as wild animals thus being dangerous.

Each member in our civilisation has to follow many rules & regulations & laws depending on contexts and his/her roles as citizen, husband/wife, father/mother, driver, employee, etc. Those rules and laws are wrapped as, usually, time-bound contracts. Just using a taxi is a short-time contract with its rules for a passenger and a driver.

IoT as cyber-physical systems must follow some rules & regulations & laws to become a very useful member of our civilisation. The famous example of such laws is “The three laws of robotics”.

Let us apply this practice of contract-based rules & regulations & laws to the Internet of Things – let us teach Things to follow their contracts thus domesticate Things.

To behave correctly, the IoT needs following digital contracts ( see “Digital-contract-as-a-process enables business in the digital world” http://improving-bpm-systems.blogspot.ch/2016/07/digital-contract-as-process-enables.html ). A digital contract is an explicit and machine-executable process between several business-parties, primarily, Things, Services and People.

2 External digital contracts

For example, a future household fridge will have, as minimum, five types of external contract simultaneously:

• with People who are living a particular household;
• with a producer of this fridge;
• with a service company for maintenance of this fridge;
• with some online shops to order various food, and 
• with some other Things within a particular household to achieve together some goals of energy consumption.

The fulfillment of some of those contracts requires the usage of the Internet. Thus, the Fridge must be able to “demonstrate” to the in-house network Router that the Fridge has rights to exchange data with some Internet-based services. Any data exchange with other internet-based services will be prohibited by the Router.

3 Internal digital contracts

In addition, being a cyber-physical system, a Thing must follow many internal contracts. Governance of all software components must be carried out as contracts for requesting a change, approval of a change, etc. Actually, all the typical IT governance and operations processes are already well-defined in COBIT, ITIL and IT4IT. They, being designed for IT departments, have to be scaled-down to the needs of Things. Even a minimalistic patch-management processes will be a huge improvement.

4 Implementation considerations

The implementation of digital contracts can be simplified by the blockchain technology (as the best, so far, records storage) which provides integrity and traceability (see “Beauty of #blockchain - doveryai, no proveryai (trust but verify)” http://improving-bpm-systems.blogspot.ch/2016/10/beauty-of-blockchain-doveryai-no.html ).

All the digital contracts, separate tasks, software components, messages, documents, workflows are notarized by blockchain.

Considering that, capabilities of particular Things may be rather different, some kind of a “majordomo” Service may be necessary to execute various digital contracts; Things will be participants in their workflows.

Also, in complex households some coordination between various digital contracts must be carried out (e.g. no preventative maintenance during receptions). This is a natural job for a “majordomo” Service. Obviously, it has its own digital contracts with the People who are living a particular household.

5 Conclusion

The proposed use of digital contracts, explicit governance and blockchain can make an impression that it will increase the complexity of IoT. Fortunately, this is not correct, because although more components will be necessary, the links between them become explicit.

In accordance with the Cynefin framework (https://en.wikipedia.org/wiki/Cynefin_Framework ), explicit linking allows progressing:

- from “Complex” situation (in which the relationship between cause and effect can only be perceived in retrospect, but not in advance)

- to “Complicated” situation (in which the relationship between cause and effect requires analysis or some other form of investigation and/or the application of expert knowledge).

Of course, a lot of painful standardisation and regulatory work is necessary ahead, but, in accordance with a Russian proverb “volkov boyat'sya — v les ne khodit'”, no pain no gain.

Thanks,
AS

Beauty of #microservices: part 9 explicit coordination as a microservice

1 Introduction

This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html

See also the previous blogposts of “Beauty of #microservices” series

• “Beauty of #microservices: part 1 #entarch #apparch” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-1-entarch.html
• “Beauty of #microservices: part 2 architecting for change” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-2.html
• “Beauty of #microservices: part 3 employ #BPM to tame the service granularly beast” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-3-employ.html
• “Beauty of #microservices: part 4 from agile development to agility of enterprise systems” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-4-from.html
• “Beauty of #microservices: part 5 defragmentation of enterprise data model” http://improving-bpm-systems.blogspot.ch/2016/09/beauty-of-microservices-part-5.html
• “Beauty of #microservices: part 6 managing state is a teamwork http://improving-bpm-systems.blogspot.ch/2016/09/beauty-of-microservices-part-6-managing.html
• “Beauty of #microservices: part 6 managing breaking the monolith” http://improving-bpm-systems.blogspot.ch/2016/09/beauty-of-microservices-part-7-breaking.html
• “Beauty of #microservices: part 8 dump-pipes & smart-containers & minimalistic-microservices” http://improving-bpm-systems.blogspot.ch/2016/09/beauty-of-microservices-part-8-dump.html

2 Things work better when they work together, on purpose (from www.tetradian.com )

To be efficient, their work must be explicitly coordinated. Certainly, this is strongly applied to all the microservices comprise an application. Of course, it is considered that an application is a several very loosely-coupled clusters of microservices to be coordinated (for example, each such a cluster is responsible for the lifecycle of a particular business entity).

Although there is an opinion that “Service is not comprised of other services due to the independence requirement” (see https://www2.opengroup.org/ogsys/catalog/W169), it is considered that some (with bigger responsibility) microservices can be assembled from other (with smaller responsivity) microservices.

There are several techniques to implement coordination.

Orchestration

• nature: centralised at design-time and centralised at run-time thus may be explicit
• specific: there is a misconception that it uses only synchronous communication (à la RPC) although it may use also asynchronous communication (à la message-passing)

Choreography

• nature: decentralised at design-time and decentralised at run-time thus implicit
• specific: uses only asynchronous communication (à la message-passing)

Reactive streams and runnable graphs

• nature: decentralised at design-time and centralised at run-time thus implicit
• specific: optimised for high volume event processing

Business-process-based

• nature: centralised at design time and decentralised at run-time thus explicit
• specific: each case is a completely separate instance with its own lifecycle; and the process may be another microservice

3 Implementation of business-process-based coordination

Of course, it should be a DSL to define an explicit coordination (e.g. BPEL, BPMN, etc.). Using the terminology from the section 7 of [REF1], a DSL-processor may act as a specialised container for DSL-scripts. Also, some microservices which are coordinated by a DSL-script may use some specialised containers. For example, a specialised container for human-operations, a specialised container for business rules, a few specialised containers for automated-operations.

4 Conclusion

Some advantages of the business-process-based technique:

• Assembled microservices have no routing logic (thus they follow SRP). 
• All the necessary microservices (assembled and dependent) can be instance-bound (help to predictive analytics).
• All the necessary microservices (assembled and dependent) can be instantiated on demand (this minimises DEVOPS).
• A particular instance may be stopped for the error-recovery without influencing other instances (operational isolation).
• A few versions of the same coordination (i.e. business process) may co-exist (versioning is easy).
• Different instances of the same process and their may be executed on different nodes (linear scaling out).
• Easy to visualise for the business people.

Thanks,
AS

Beauty of #microservices: part 7 breaking the monolith

1 Introduction

This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html

See also the previous blogposts of “Beauty of #microservices” series

• “Beauty of #microservices: part 1 #entarch #apparch” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-1-entarch.html
• “Beauty of #microservices: part 2 architecting for change” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-2.html
• “Beauty of #microservices: part 3 employ #BPM to tame the service granularly beast” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-3-employ.html
• “Beauty of #microservices: part 4 from agile development to agility of enterprise systems” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-4-from.html
• “Beauty of #microservices: part 5 defragmentation of enterprise data model” http://improving-bpm-systems.blogspot.ch/2016/09/beauty-of-microservices-part-5.html
• “Beauty of #microservices: part 6 managing state is a teamwork http://improving-bpm-systems.blogspot.ch/2016/09/beauty-of-microservices-part-6-managing.html

2 Breaking the monolith

This blogpost is a continuation of two previous ones:

• "Enterprise patterns: eclipse" http://improving-bpm-systems.blogspot.ch/2013/06/enterprise-patterns-eclipse.html
• "How to modernise a legacy ERP" http://improving-bpm-systems.blogspot.ch/2014/04/ideas-for-bpmshift-delenda-est-vendor_27.html

Below, a series of steps to show how to remove from a monolith (actually a home-made ERP) some functionality around a particular Business Entity (BE) or a group of BEs.

At the AS-IS step, the monolith is the master of everything related to this BE (thisBE) – data, rules, processes and events (which are generated during the lifecycle of this BE and may affect other BEs.)

The first step is to externalise and make explicit the process to manage this BE in a BPM-suite tool. Keeping of thisBE (i.e. its data) is externalised as well. Also the associated rules must be externalised (as a copy) to reproduce the business logic spread in the monolith.

The monolith keeps its slave copy of data which are maintained via some stor-API. The associated business logic and event logic are still managed by the monolith. The data (as a slave) must always stay in the monolith because they may be used somewhere.

The second step is to externalise rules (when they are good enough to cover all the existing rules from the monolith).

At the TO-BE step, everything related to thisBE is externalised, but the associated events must be “injected” into the monolith by some func-API.

3 Anti-pattern DOUM

Avoid the "DOUble Master" (DOUM) anti-pattern (section 5.6 of my book "Improving enterprise BPM systems" http://www.improving-bpm-systems.com/book ).

As coordination can be carried out by an application or by a process engine, we have to be very careful to avoid the “double master” anti-pattern. At any moment in time there must be only one master responsible for the coordination of a particular process instance. (Of course, the coordination role may be delegated if appropriate.) This is analogous to a well-organised meeting where the chairperson decides who talks next.

The non-recognition of this anti-pattern can be very costly. We have observed a BPM solution which allowed the modification of data by a process engine, by an interactive application (i.e. by a human) and by a batch at the same time. The coordination of activities was based on data and, if necessary, the application or the batch could “correct” the process. The process engine was used mainly for the handling of three human activities, and the implementation of this solution (for a relatively simple business process) took several man-years.

4 Conclusion

How will you eat an elephant? Piece-by-piece, of course.

Thanks,
AS

Beauty of #microservices: part 5 defragmentation of enterprise data model

1 Introduction

This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html which uses other blogposts about microservices http://improving-bpm-systems.blogspot.ch/search/label/%23microservices

See also the previous blogposts of “Beauty of #microservices” series.

• “Beauty of #microservices: part 1 #entarch #apparch” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-1-entarch.html
• “Beauty of #microservices: part 2 architecting for change” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-2.html
• “Beauty of #microservices: part 3 employ #BPM to tame the service granularly beast” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-3-employ.html
• “Beauty of #microservices: part 4 from agile development to agility of enterprise systems” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-4-from.html 

2 Unfortunate historical fragmentation of enterprise data

In modern enterprise computing environments, an enterprise data model (as a set of business-entities defined at the enterprise level) is, usually, an utopia. Enterprise data are spread among many existing applications that master some attributes of some business-entities. Also some business-entities are duplicated (usually, partially, by some fragments of them) among existing applications.

Because each of those applications (in-house developed, SaaS-based, PaaS-based, intact COTS, modified COTS) has its own lifecycle, the enterprise data model is far from the reality and complex data integration is mandatory to keep the integrity of the enterprise data.

Each new application has its own data and uses data from other existing applications. Thus, an application data model comprises some views on some business-entities from the enterprise data model. However, in such an application data model only some attributes are under control of the application.

If an application has to use data from several other applications then the evolution of some application may have a destructive effect on some other applications – typical “Château de cartes” anti-pattern.

3 Enterprise data model must be flexibility-driven not legacy-held

Ideally, an enterprise data model must

• easy to evolve (by definition)
• be service oriented so data can be accessed only via API (this its fragments can be joint up together even not sitting in the same database)
• implement total versioning for some business-entities (to cover the enterprise lifecycle)
• cover all the types of business-entities
  • transactional data (your business-specific and business-critical data)
  • reference data (taken from other sources and business-parties)
  • operational data (about how the work has been done)
  • reports
  • analytics data
  • records
  • documents
  • media
  • social

There are several techniques to move to this ideal.

Make some functionality enterprise-wide as a Corporate Unified Business Execution (CUBE) platform (see http://improving-bpm-systems.blogspot.ch/2015/10/enterprise-patterns-peas-example-cube.html ). Thus an individual application comprises only business-specific functionality. For example:

• security management (identity, authentication and authorization)
• content and knowledge management
• software factory
• reporting and analytics
• business process management
• records management
• etc.

Make explicit data update processes – see the pattern “Practical process patterns: Synchronisation Of Sources (SOS)” http://www.slideshare.net/samarin/practical-process-pattern

Use some specific characteristics of your enterprise.

4 Adding flexibility

The classic “point-to-point” scenario is that a solution can read some data from some applications via storage-centric API or asset API or stor-API. Actually, those data are application-specific views on some enterprise business entities, e.g. “ClientERP BE” is the business entity CLIENT as it is defined in an ERP. Of course, in this sensation, the solution 1 strong depends on the evolution some other applications.

Another classic “data-access-layer” scenario is to use a microservice aggregate all the application-specific views in a read-only business entity.

The ability of microservices to have their own persistence store (which may be just a table in a common database) is very useful to implement a small extension of an business-entity. The same technique may help to avoid customisation of COTS products. This is a safe way to mode to an ideal enterprise data-model.

With some efforts (mainly for externalising some business logic), read-only APIs may be transformed into read-write APIs. If a business-entity uses more than one underlying applications then a “data update process” is necessary to implement a multi-phase commit.

If it is possible to externalise the business logic then some functionality-centric API or business API or func-API may be implemented.

5 Conclusion

With the use of microservices and other technologies, an enterprise data model may:

• become always up-to-date and
• evolve with the speed of agile development.

Thanks,
AS

Beauty of #microservices: part 4 from agile development to agility of enterprise systems

1 Introduction

This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html which uses other blogposts about microservices http://improving-bpm-systems.blogspot.ch/search/label/%23microservices

See also the previous blogposts of “Beauty of #microservices” topic.

• “Beauty of #microservices: part 1 #entarch #apparch” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-1-entarch.html for the definition of the following terms: process, orchestration, microservice or Autonomous Component (AC), and microservice architecture. 
• “Beauty of #microservices: part 2 architecting for change” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-2.html
• “Beauty of #microservices: part 3 employ #BPM to tame the service granularly beast” http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-3-employ.html

2 Agility during the whole application lifecycle

This blogpost is named after my 10 years old presentation “From agile development to agile evolution of enterprise systems” ( see http://www.slideshare.net/samarin/from-agile-development-to-agile-evolution-of-enterprise-systems ) with a small change to introduce the word “agility”.

This done because of one of Matthew Kern’s blogpost “Fixing Agile with Architecture” (see https://www.linkedin.com/pulse/fixing-agile-architecture-part-4-matthew-kern-cea-cissp-issap-pmp- ). Obviously, architecture works with a longer timespan, e.g. the whole application lifecycle ( see also http://improving-bpm-systems.blogspot.ch/2014/03/enterprise-patterns-ear.html ) and application development covers only the delivery of the first production version. Thus architecture must guide the application development.

Considering that the TCO of a custom-developed application is only 20% for the implementation of its first production version then the rest 80% must be taken seriously. A real-life example: the cost for a small modification (changing an UI field from free-text to drop-box) in a custom-developed ERP is 20 KCHF and the implementation time is 3 months.

The agile development does guarantee that prototyping and implementation of a new application can be done rather quickly. But, the agile development does not guarantee that the production version of an application can be changed very quickly. Just compare – change a wheel of a parked car and change a wheel of the moving car.

The table below compares the level of frequency of changes for various application building blocks ( see the part 3 -- http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-3-employ.html ) during different phases of the application lifecycle.

Building blocks with the low level of changing frequency may be kept as microservices, services or even be embedded into a monolith. Building blocks with the medium or high level of changing frequency must be kept as microservices.

Thus the latter can be changed quickly and safe (without damaging the running application) because each new version of microservice assemble is a different process-template (a formal description of the process). Each process-template may be enacted as several process-instances.

The distinction between process template and process instance is very important. The lifecycle of process-template is controlled at design-time. The lifecycle of process instance is controlled at run-time. A process instance is created, maybe suspended & resumed and finally terminated. Many process instances from various templates may co-exist at the same time as shown below.

3 Conclusion

Microservice architecture, together with BPM, resolves two fundamental problems of the agile development:

1. building blocks (which are discovered by BPM) are to be implemented à la agile development and 
2. some building blocks are to be implemented as microservice thus enables agility of enterprise systems (quick and safe changing of production applications).

Thanks,
AS

Beauty of #microservices: part 3 employ #BPM to tame the service granularity beast

1 Introduction

This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html which uses other blogposts about microservices http://improving-bpm-systems.blogspot.ch/search/label/%23microservices

See also the previous blogposts of “Beauty of #microservices” topic.

• “Beauty of #microservices: part 1 #entarch #apparch” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-1-entarch.html for the definition of the following terms: process, orchestration, microservice or Autonomous Component (AC), and microservice architecture. 
• “Beauty of #microservices: part 2 architecting for change” http://improving-bpm-systems.blogspot.ch/2015/05/beauty-of-microservices-part-2.html

2 Service granularity

A recent article from Zhamak Dehghani of ThoughtWorks https://dzone.com/articles/taking-microservices-home says “Service Modeling is Difficult” and “Find your Granularity”.

Of course, the service granularity is difficult, because it is usually done bottom-up, i.e. in the solution-space without systematic understanding of the problem-space.

What could be a good enough description of a particular problem-space? Ideally, the following is necessary to analyse objectively a particular problem-space:

1. An domain-of-interest (e.g. business unit, group of processes, etc.) whose unsatisfactory behaviour is qualified as a problem.
2. An explicit and machine-executable model of this domain-of-interest.
3. Data those are sufficient to simulate the behaviour of the domain-of-interest with its model.

To build an explicit and machine-executable model of a particular domain-of-interest, this domain-of-interest must be considered as a system. Considering the following definition of the concept ‘system’ “set of interacting discrete parts organized as a whole which exhibits (as the result of interaction between the parts) some emergent characteristics indispensable to achieve one or more stated purposes”, those “discrete parts” have to be identified first. In other words, it is mandatory to find something that will allow to state: “this domain-of-interest is a system of ”.

Typical business artefacts that are used to describe a particular business are: capabilities, functions, services, processes, roles, data, etc. Is it correct to state that a domain-of-interest is a system of capabilities? System of functions? System of services? Not sure – they (capabilities, functions or services) can be explicit, but they can’t form a machine-executable model to simulate the behaviour of domain-of-interest.

So far, it is correct to state that a particular domain-of-interest is a system of business processes ( see http://improving-bpm-systems.blogspot.ch/2014/03/enterprise-as-system-of-processes.html ).

Each business process is an assembly of the following artefacts: events, human activities, automated activities, services, processes, data, documents, rules, roles, records (audit trails), reports and KPIs. All those artefacts are designed in accordance with their domain because they are specific for a particular process or a group of processes. Thus, domain-driven design is automagically followed.

For example, data structures comprise some data necessary for a particular activity and those data are sub-sets of an enterprise data model (extra blogpost will be written on this). Roles are around a particular activity – who can execute this unit-of-work, who can supervise it, who can consult it, etc. Those roles are composed from the enterprise-wide organisational and functional roles.

All those artefacts are units-of-functionality with single responsibility.

All those artefacts have well-defined interfaces.

The majority of those artefacts are stateless.

Thus there is only one step away from the microservice architecture – just wrap those artefacts as services and deploy them separately.

Some of those artefacts in some BPM-suite tools are already microservices! In other BPM-suite tools, some of those artefacts are in the same JVM (see “programmable monolith” in http://improving-bpm-systems.blogspot.ch/2015/12/typology-of-platforms.html ).

3 Conclusion

The use of BPM (which is a top-down analysis of a domain-of-interest) resolve the following difficulties of the microservice architecture:

• granularity of microservices is defined in accordance with the domain-of-interest, thus “domain-driven design” is achieved
• relationships between microservices become explicit and they are known at the design-time
• the number of relationships between microservices is reduced thus complexity is reduced as well

Thanks,
AS