Functions Required in the Cloud PaaS Layer to Support SOA

The Platform as a Service (PaaS)

Whether a Private or Public Cloud is built internally, or contracted for, to support SOA, its the "Platform as a Service (PaaS)" layer will be required to have certain functions essential for the support of SOA-based Composite Applications/Services.

The PaaS Layer is "the second" layer of the US National Institute of Standards and Technology's (NIST) Cloud Computing Architecture.  It is the middle layer of the architecture, between the Software as a Service (SaaS) and Infrastructure as a Service (IaaS).  In fact, the PaaS is the first complete actual layer (as opposed to virtual) of the Cloud Computing Architecture.  The only actual functions supported by the SaaS are the user interface functions, which will include the security interface(s) (see my post SOA, the User Interface Service Components, Apps, and Validation before Registering).

The PaaS contains all of the functions needed to run the applications (Services), as well as the actual repository containing the code for the application--for Services, the Service Components.  For a SOA-based set of applications, there are two functions that they require in the PaaS.  First, is the Service Component Registry.  The Service Component Registry is a virtual catalog of the location of Service Components.  It enables the orchestration and choreography processes to find the Service Components, which create Composite Applications.

Second, the PaaS must have Orchestration and/or Choreography Engines with associate Rules Engine and Rules Repository.  I have discuss the difference between orchestration and choreography in my post "SOA Orchestration and Choreography Comparison".  These functions (engines) are based in the PaaS.  The rules engine and rules repository enable and supports automated business rules,which, in turn, enables and supports the organization's policies and standards.

Private versus Public SOA Cloud Architecture

As commonly used, a private cloud is a system within one ownership domain; while a public cloud is a system that crosses ownership domains.  In the language of SOA, private cloud is an Enterprise SOA, while a public cloud is an Internet SOA.  As I discussed in the post on orchestration and choreography, Enterprise SOA uses orchestration (mainly) to execute its composite applications, while Internet SOS always uses choreography.  However, I suspect that most Enterprise SOAs will include both orchestration and choreography, as they will use Service Components outside their ownership domain for occasional ad hoc composite applications; for such things as research and risk reduction.

Another key difference between Private and Public SOA Architectures is that an Enterprise (private) SOA will use an application layer communications system called an Enterprise Service Bus (ESB) to enable the Service Components of the Composite Application to communicate.  On the other hand, the Internet (public) SOA will use the Internet to enable the Service Components to communicate.  For the Cloud Architecture this means that the PaaS of the private cloud will include ESB functionality, while the public cloud will use the interconnectivity of the Internet provided by the IaaS for these functions.

Using Enterprise Architecture to Reduce the Federal Debt

The Federal debt is caused, in part, by laws and regulations that are cumbersome, obsolete, and conflicting with other laws and regulations and with the charters of the enforcing departments and agencies.  These are the "Catch 22's" that infect all governments.  This problem makes the departments very cost inefficient.  In my experience as a process engineer and from observations of organizations in operation, an organization uses a minimum of 10 percent of its resources in causing and fighting this process friction (sometimes it can be much more). 

There have been many times when the US Federal government has "addressed" this cause with one time programs over the years, with varying degrees of success.  The unfortunately the operative phrase is "one time".  To be successful, all laws, regulations, policies, and standards have to be structured and there must be a process to revisit each to ensure that the are measurably performing as expected.

If properly used an Enterprise Architecture process and repository in the form of the Federal Enterprise Architecture Framework (FEAF) can reduce or minimize the conflicts among laws, regulations, policies, and standards over a 2 to 10 years period--depending on the size of the organization.  This does not mean that there will be no benefits for 2 to 10 years, in fact management may notice some within 3 to 6 months.  What it does means is that the complete transformation and acceptance of the the Enterprise Architecture process to help with investment and policy decision-making will take that long to be fully accepted by and inculcated into the organization.  Unfortunately, without a champion at the top of the leadership, working with a top-notch transformational enterprise architect, it will never happen...due to the many reasons, excuses, and loss of influence by various levels of management and their constituents.

A second cause is that the departments, bureaus, and agencies are running more on their management's agendas than on the organizations' charters.  This means that a single organization may have many duplicative processes procedures and tools, which greatly reduces the cost efficiency of the organization.  These are cause by private managerial agendas.  Private managerial agendas are caused by conflicts of policies/regulations, the NIH (that is, the Not Invented Here) syndrome, or the manager's personal empire building.

Sometimes the managers' private agendas are caused by conflicts in or obsolesence of regulations and rules preventing the managers from achieving their mission.  A good Governenace and Policy Management process coupled with a good Enterprise Architecture process will greatly reduce this cause for private agenda's.

Other times, the sub-organizations of an organization will decide that the organization does not understand their requirements for a particular function and therefore, they feel the need to create their own. Or, it may simply be that they do not trust the organization designated to perform the function to perform it in the manner they would prefer, and therefore, it needed its own.  A more malicious version of this, is managerial empire building, where the manager wants to enhance his or her own status by controlling all of the functions required to perform a given process, and additionally, takes a very wide interpretation the his or her mission or charter, and strategies, policies, procedures, and tooling.

In all of these instances the manager has a case of the NIH syndrome; which causes much waste of the organization's resources.  Enterprise Architecture can ameliorate the cause by mapping all strategies and processes to the organization's mission as recommended by the FEAF, providing measurable line-of-sight from the mission to the processes and tools.  For example, one in US Federal department, one organization had been charter to perform the reservation function for the entire department.  As of 2007, team of Enterprise Architects had found at least 50 sub-organizations with independent reservation functions.  Simply, by consolidating many of these functions into a single system, the department was able to save 10s of millions of dollars per year in development and operating costs.

A third cause is "Political Correctness" (PC).  This boils down to various political constituents getting a piece of the pie, irrespective of their knowledge, skills, and capabilities.  Part of this is due to organizational networking (which "PC" lobbists characterize as the "good ol'boy" network--which may have some validity).  DARPA, NSF and other Federal R&D organizations tend to award contracts to organizations that have performed well in the past--not a bad way to chose.  But, it leads to these agencies overlooking organizations with no track record; thus, they may miss good ideas.  Therefore, it stratifies organizations, giving organizations with a track record an advantage--their abilities are known--over other organizations--whose abilities are unknown.

Therefore, the US Congress, in its inimitable wisedom mandated that various "special constituents" get a piece of any large program irrespective of competence to ensure "fairness", and "equality."  However, requiring all large programs, especially those with high knowledge and R&D skills to include "PC" firms simply to win the contract, whether or not these firms have any skills at all, is very costly and frequently causes major cost overruns and program failure.

While Congress's objective was noble, the unintended consequences causing negative externalities have been very expensive to both the government and the long-term ability of the United States to compete in the global economy.

Layers of management, problems of coordination, and private agendas among the "teaming" partners--each vying for a larger piece of the project, even after it's been awarded.  When these "disadvantaged" firms fail to perform, there is a major risk of a cascade affect, that without quick transference or mitigation cause the program to fail.  But, process of transferring is tangled in contractual and inter-organizational political problems that slow the process to a creep or stall it altogether.

The "PC" regulations together with their affects creates dysfunctional Virtual Extended Enterprises (VEE).  Note that VEE's are business versions of SOA's Composite Applications, in that they are composed of many small, but highly skilled and knowledgeable organizations.  Through the life cycle of a product these component organizations can be introduced into and let go from the VEE in a manner similar to that of Service Component replacement for purposes of technology insertion.  The "PC" regulations militate against these changes, creating a high risk of failure for programs trying to produce unprecedented complex systems.  This adds greatly to waste of Federal resources.

Enterprise Architecture can provide a method for sorting out the best and brightest organizations from both "advantaged" and "disadvantaged" but incompetent organizations, if the enterprise architecture processes and repository are properly organized and the processes are correctly executed.

Longer term, this waste of resources from all of these causes will cause the US to both lose its competitive edge militarily and economically.  Again, over time, the proper use of the enterprise architecture processes and repository will enable a competitive advantage.

Also see the post entitled "Thoughts on Solving the Federal Debt Issue", and "Rebuilding Jobs in the United States".

Assembling Services: A Paradigm Shift in Creating and Maintaining Applications

Technically, the key difference in Service Oriented Architecture (SOA), from other IT architectures is that it assembles the applications using the process flow as the guide.  This requires separation of the functions from the process or work flow.

There are many advantages in splitting process flow from the functions.  Among these are:

1. Agility of the application - Simply by separating the functions from the process flow and "coding" them separately the application becomes more flexible.  The reason for this is that simply by changing the process flow (reordering the functions) the application (the service) becomes, affectively, a new application.  If the process flow engine is integrated with a rules engine and repository, and if the rules link directly to the organization's policies, then changing a policy or standard will also, affectively, create a new application (service).  Since neither of these changes requires an additional or recoding, then time to respond to a challenge or opportunity is greatly reduced; making application much more agile (for the definition of Agility see the post "What is Agility?"
2. Ability to insert new technology -  Because a service is assembled from a set of objects or "small applications" - also called Service Components - into a Composite Application (which is the code for the service), one function (one Service Component) can be upgraded without touching the rest of the application.  This greatly simplifies the ability to upgrade the application, also increasing its agility.
3. Enables effective modeling of the process flow - Because the process flow is separate from the functions, the System Architect can model both the current and any proposed or candidate process flows to better determine the optimality of both the process and the enabling and supporting service (Composite Application).  If this "(business) process modeling" is properly linked into the Service development and maintenance processes of an organization, it will greatly aid in optimizing the services even with unexpected challenges and opportunities.
4. Enables good policy enforcement - If, as noted above, the process flow engine is linked with a rules engine and repository, then there is "an automatic" enforcement point each time the process flow engine evokes a new function.  Additionally, when a rule changes, it is immediately reflected at the enforcement point in the process flow engine.
5. Measurement to ensure that the SLA's are met - Finally, since all Services use the process flow engine, it is simple to measure the throughput of each Service Component (function).  This enables good measurement of both the Business-SLA's and the OLAs, simpler root cause analysis when the SLA's are not met, and to feedback to the modeling of the process to help process optimization.

Asset and Enterprise Architecture Repository

In the near future, all economic and governmental organizations will need an Asset and Enterprise Architecture Repository to survive.

An Asset and Enterprise Architecture Repository is the content store for all data and information related to the organization's current assets (structured into an "as-is" architecture) and its "to-be" architecture.  The term assets might be limited to the organization's physical assets, but can additionally include its processes, strategies, mission, vision, and personnel (attributed with their skills and talents). 

In the case of SOA, the enterprise architecture must include its processes and might include its vision, mission, and strategies.  For organizations intent on using SOA, these inclusions shift the responsibilities for the processes  of the organization to the IT sub-organization and make Enterprise Architecture, as a decision-making process very important: a major process paradigm shift.