Australian retail playing catch up

Almost every day now I am drawn into conversations about how much cheaper common consumer products are to buy online. The following is a list of products my immediate circle of friends has bought online from overseas (mostly from US and UK web stores) in the past month – books, shoes, DVDs, dresses, tailor made shirts and suits, camera accessories, toys and computer parts.

The price differences are astounding. Books delivered to your door from the UK for $8 when local bookstores are asking $24 for the same book. Shoes that retail locally for $200 are just $100 delivered from the US. It seems that if you can’t find online prices for less than 50% of the local retail price you just aren’t trying.

The Australia Institute (TAI) conducted a survey of online shopping and has some great examples of the price differences between local and online foreign retailers. A Sony Bravia television is half the price to buy online from the US ($995 instead of $1999), DVDs are usually half the price online, and even high end bicycles are half price if you get them delivered from the UK ($1599 instead of $2999).

Even more bizarrely my local bottle shop has a six-pack of imported German beer for $10 – that’s $4 less than the XXXX that is delivered a mere two kilometres from the brewery (and yes that is the brewed in Germany beer, not the locally brewed German style).

And one thing that really bugged me was that at the Hofer supermarket in Vienna a couple of years ago - Australian wine was 2.5euros a bottle, but the same wine here is more than $10.

What is going on!

We have a situation where not only are Australian products more expensive locally, but so are imported products!

My theories are -

1. Adjustments to higher exchange rates are very slow. If the dollar appears to stabilise around $1.05USD we can expect local prices to slowly converge to international prices.

2. Australian retailers are simply years behind their European and US counterparts in terms of adopting more efficient business models (think Aldi, Ikea),and especially for online retailing. Again, we can expect this to change slowly.

3. Australians still feel wealthy and don’t feel compelled to hunt for the best deals. This is changing as the TAI survey showed.

4. Many have suggested high rents for well located retail space as part of the explanation. This might be a contributing factor but I imagine that the trend for retail rents is down.

5. People get upset when prices fall. Remember the milk wars, the beer wars?

6. Local producers have no incentive to sell locally below the price they receive from export markets (as I discussed here).

I’m open to any other suggestions, but there really doesn’t seem to be one factor to explain this retailing discrepancy. I also believe that the price differentials between local and foreign online retailers are likely to shrink from now on.

Governance, and Policy Management Processes: the Linkage with SOA

Business Rules and Process Flow

In a recent post, A Model of an Organization's Control Function using IDEF0 Model, The OODA Loop, and Enterprise Architecture, I briefly described the Governance and the Policy Management processes within the context of the IDEF0 Model of the organization and the OODA Loop process pattern.  One function of the Policy Management process is to create "business" (organizational) rules and measurably instantiate the policies and standards.  Once created, these rules can be automated since they turn out to be the "if-then-else" statements in the process logic (for more on this see Types of Business Rules).  As discussed in the post, there are at least three types of Business Rules, knowledge rules, event rules, and value rules.  Each of these rules is an attributed metric, that is, it measures what is going on or changing and compares that with some limit or standard.  This makes the rules relatively easy to automate.

SOA and Process Flow

Many individuals and organizations still confuse Service Oriented Architecture (SOA) with Web Services on either the Internet (the Public Cloud) or an intranet (a Private Cloud).  SOA is a much larger change in IT architecture than that (i.e., "A gaggle of Web Services does not a SOA make").  As I discussed in SOA, The (Business) Process Layer, and The System Architecture , one key difference is that SOA formally separates the process flow from the functions of the process.  This difference makes it feasible to enforce a continuously changing set of rules while the system is operating rather than updating or developing new applications then rolling them out.  Consequently, this is one of the reasons that SOA-based systems are very agile and one of the reasons that good Governance and Policy Management processes are so important to the success of SOA-based systems as Gartner Group and other studies of SOA implementations have found. 

Additionally, it is the reason that both the Orchestration and Choreography engines of the SOA-supporting infrastructure must be directly linked to a Rules engine, which uses a Rules Repository (within the AEAR) since both control the process flow of the Composite Application.  In this configuration any changes in a rule will immediately affect all processes using the rule (so rules modeling, verification, and validation is an imperative as functions of the Policy Management Process or the Services will end up in a complete).

Where Rules Apply
There is yet another complication in this rules management discussion, relative to SOA, System, and Enterprise Architecture, that is some policies and standards apply during the development or transformation process (Design Time) and other apply during Service operation (Operational).  All Design Time and Operational Policies and Standards are Design Constraints (see Why Separate Design Constraints from the Customer's System Requirements? for a discussion of Design Constraints).  Both the Systems Engineer and the System Architect must ensure that these requirements are identified and met.

Design Time

Design Time policies and standards divide into two types, those that effect how the designer and developers implement the Composite Application and those that effect the resulting Composite Application.  For example, in a SEI CMMI Level 5 organization, developing a Composite Application following the dictates of the CMMI Level 3 process is a must, but, whether or not the Composite Application has two-factor authentication is not a concern.  However, if the organization has an organizational standard that all applications will have two-factor authentication will directly affect the resulting Composite Application.  So the first example standard effects "how the Composite Application is built" while the second dictates "what is built".  Good process engineering and management, systems engineering, and system architecture will ensure that all of these requirements are met.

Operational

Operational Policies and Standards are not IT-based, but "business" based for whatever the "business" of the organization is.  They are the attributes of the "If-then-else" branching clauses in the "business" process flow, which is instantiated in the code.  While the branching statements themselves will, generally (See the sidebar for the caveat), remain the same, the attributes, as instantiated by the rules can and will change.  This makes Verification and Validation of the Service (Composite Application) process flows during the development or transformation process difficult, at best.  This is the reason for the modeling, verification, and validation of the Services affected by the rules change before it is roll into the operational environment.

Sidebar
In a four part paper in the Journal of Enterprise Architecture, I envisioned the Self Adapting Service Architecture in which both the form and attributes of the branching statements adapt to changes as part of self-rules setting (or information abstraction) process (see my post The Hierarchy of Knowledge and Wisdom for the definitions).  A prototype of such a system is the IBM's Watson computer that won at Jeopardy over several Jeopardy champions.  As I suggested in the Fourth part of the paper, I would not expect such systems to be operational before 2021.
End of Sidebar

The Linkage
From this brief discussion, I think I've made clear that there is a close linkage between SOA and the Policies and Standards set by the Governance and Policy Management processes.  Getting this linkage in place will require the teaming of the leadership, Enterprise Architect, management, Systems Engineers, System Architect, and Subject Matter Experts (SMEs); this is a significant cultural shift throughout the organization, but organizations that achieve it will have a major competitive advantage over those that don't.

Dwelling finance springs back

The ABS released their April dwelling finance data today, and there was quite a bounce for owner occupiers across all States, but investor finance continues to fall.

Taking a look at the big picture it is hard to know whether this one month's data is particularly meaningful.

Great Stagnation?

Tyler Cowen has an ebook that presents his hypothesis that America is undergoing a great stagnation. What he means is that teh rate technological change and economic growth has slowed since about 1973. You can get most of his message from the TEDx talk in the below video.

While Cowen acknowledges the great leaps in communication technology, I feel his presentation glosses over a lot of medical technology which is highly valuable and has continued to improve life expectancy.

He also glosses over a lot of other changes that people value but don't get recorded in the statistics (for example greater equality of genders and races or lower crime rates). The more effort society directs towards these social advances, the less effort it can direct towards technological marvels.

Overall it's a very interesting video for anyone curious about economics and modern history.

SOA Architects: "Don't Bury the Business Rules in the Database"

Why Rules Get Buried in Databases

Even though the Millennium Bug was first noted in 1982 (or perhaps earlier), it was not widely recognized until the mid-1990s.  Fundamentally, the "millennium bug"was caused by the need to minimize the amount of data storage during the 1960s and 1970s and the lack of understanding of System Architecture earlier.

Fragmented Architecture

Initially, the business logic was hard coded into the programming.  Partially, this was due to the fact that in the 1950s and early 1960s, computers only assisted in supporting the mathematical (e.g., accounting) functions of an organization.  So, it was rather straightforward to code in the Business Rules (the if-then-else, as I discuss in Types of Business Rules).  However, as the System Developers attempted to integrate these functions and as the organization's leaders attempted to respond to a changing organizational environment and to changing governmental regulations, they found that changing the enabling and supporting IT systems difficult and expensive (And many large organization's still do, since they are still dependent on millions of lines of code written in the 1960s and early 1970s).  A great part of the reason for this lack of agility was and is the hard-coded business rules in the program's logic--some of which is documented only in that logic.

Monolithic Architecture

In the early 1970s, as disk storage became somewhat less expensive, Relational Data Bases (RDBs) and Relational Data Base Management Systems (RDBMS) started to replace earlier data storage systems.  The RDBMS enabled developer to map data elements into tables with relationship among all the data.  This mapping essentially converts data into information (see the Blog's glossary page).

The advent of the RDBMS was one of the technical enablers of the MRP, MRPII, and ERP systems, built using Monolithic Architecture.  For example the SAP R3 system could not exist without an associated Oracle or other RDBMS system.  The reason is that the Monolithic Architecture is superior to a Fragmented Architecture because it reduces the number of interfaces that have to be maintained among programs to nearly zero--and as discussed in SOA, The (Business) Process Layer, and The System Architecture I found that the cost per interface was $100 per month.  When you multiple that by the 1000s or 10,000s of interfaces required in a large organization with a Fragmented Architecture, the apparent cost efficiency is enough to delight any finance engineer, be they the CFO or any of his or her minions.

Once the RDBMS systems took hold, the Database Designers (DBD) and coders found that the easiest point to validate the data input was just before data insertion into the tables.  The RDBMS suppliers responded with triggers, (if-then-else logic within the RDBMS).  These triggers rejected data that was out of conformance with some standard; which is the essence of a Business Rule.  However, once the DBD was given this tool, he or she started creating ever more complex triggers; triggers that embody Business Rules that may have little to do with data validation.  For example, many times when data is inserted a trigger will change a flag or other indicator within a set of metadata (data about the data).  The flag may then change the course of the function or business process.  This type of trigger buries the Business Rules in the database.

The problem with an application built on a Monolithic Architecture is the inflexibility of the entire system, as I've discussed in The (Business) Process Layer, and The System Architecture and several other posts.  One key technical element of this inflexibility is the System Architect and SME's inability to change or tailor the data structures because the application is dependent on a single data structure (as well as a standard process, etc.).  When the triggers in the RDBMS include a significant number of Business Rules, this increases the inflexibility because the detailed designer/coder/SME may have great difficulty in even locating the Business Rule.  Therefore, he or she must write an add-on or bolt-in to modify the Business Rule, with all of the inherent configuration management problems that creates.  Further, even if the detailed designer can find the proper trigger to modify, ERP systems do not usually identify all of the functions that use the data or the trigger, so there is a significant risk of induced defects, defects in a secondary process caused by the rules change to support the primary process or function of the transformation effort.

Service Oriented and Other Agile Architectures

As I discussed in  The (Business) Process Layer, and The System Architecture, and several other posts, Service Oriented Architecture (SOA) and other agile processes require separation of the processes and Business Rules from the functions of the process. From the discussion, above, the reasons should be obvious.  Both good Governance and Policy Management (see A Model of an Organization's Control Function using IDEF0 Model, The OODA Loop, and Enterprise Architecture, and Enterprise Architecture, Chaos Theory, Governance, Policy Management, and Mission Alignment) and Agile Mission Alignment require this separation.  Without a Rules Repository, the Enteprise Architect and the System Architect are much less effective at optimizing the SOA or other agile architecture to support the organization in a constantly changing operational and technical environment.

The Prescription: an Agile Architectural Process Pattern

As I see it, the following is a prescription for transforming an organization from function-based to process-based, and the IT architecture of the enabling and supporting systems from Fragmented and Monolithic to agile and Service Oriented is to first understand and target agility, in addition to process effectiveness and cost efficiency.  Second, define or ensure that there are formal Enterprise Architecture (Mission Alignment, Governance and Policy Management) and Systems Engineering/System Architecture processes that coordination with one another.  These processes should have a three month cycle, not a year or more, to ensure requirements agility.  Third, as I described in my posts, Initially implementing an Asset and Enterprise Architecture Process and an AEAR, and Asset and Enterprise Architecture Repository, start with the output of current projects to create the AEAR and exercise the processes.  Fourth, expect that the processes will be less than optimal at the start (the learning curve, see the Superiority Principle).  The risk reduction on this is that since the process cycle is three months, instead of a year or more, learning takes place at a much greater rate.

Addendum: When to Use Triggers When Building to An Agile Architecture
I had a futher thought about triggers.  The Gartner Group recommends segmenting the complete data structure (supporting a monolithic architecture) into databases supporting the individual Service Components (e.g., Web Service).  Each of these Service Components would support either an entire or a logical portion of an organizational function.  I would suggest that triggers can be used within the databases supporting individual functions (Service Components), especially for ensuring data completeness and correctness.

Types of Business Rules

Why the Types of Business Rules Should be a Concern
As briefly discussed in my post A Model of an Organization's Control Function using IDEF0 Model, The OODA Loop, and Enterprise Architecture, the operational result of the Governance and Policy Management Processes is Business Rules.  This post will discuss the types of business rules because a good understanding of the types of rules is an imperative in understanding how they effect an automated business process and their enabling and supporting IT systems.  In the US Federal Government, the analog of these Business Rules are the Regulations that enable and support the Laws.

The reason needing to understand the types of Business Rules is that to effectively, cost efficiently, and agilely automate the support of an organization's processes requires a formal method for enforcing and mediating/adjudicating the organization's policies and standards.  This formal method requires a translation/transformation of the policies and standards into rules that can be automated.  But there are at least three type of enforcement rules.  Understanding what each type of rule does two things; helps to ensure that "the rules" aren't muddled, and ensures that the policies and standards are adequately and completely enabled (i.e., that some part of the policy or standard is not checked because there is no rule to support it).

In addition, if the rules are stored in a Rule Repository (which is part of the Asset and Enterprise Architecture Repository, AEAR), then as new rules are put in place, the Enterprise Architect can check to ensure that they do not conflict with other rules.  And, as rules are revised/updated the Enterprise Architect can check to ensure that they do not create defects in the processes by the way they are implemented.

Types of Business Rules
A Business Rule is a metric or group of metrics for determining whether a system or user is out of conformance with a policy or standard and the change in value for the organization due to the non-conformance.  This definition may be "as clear as mud, but it does cover the ground".  To clarify the definition, let's consider the types of rules.

There are at least three types of Business Rules that serve different roles in enforcing the policies and standards within the organization's processes and systems.  

Knowledge Rules
A knowledge rule is a metric within which the process, system, or user is in conformance and outside of which the process, system, or user may be out of conformance.  The operative word is "may".  Many times it takes more than one metric for a rule "to be broken".  A trivial example is based on an old US Air Force saying, "He ran out of airspeed, altitude, and ideas"; meaning the aircraft crashed.  With sufficient airspeed an aircraft can gain altitude; with sufficient altitude an aircraft can gain airspeed; and so on.  The point is that knowledge of the minimum airspeed to fly, the advantage of altitude, and other ideas(like landing in the water or a soft grassy area are all bits of knowledge that can keep a pilot alive and an aircraft in flyable (or at least repairable) condition.  However, if the pilot violates all of them concurrently, he or she has a serious exciting short-term problem and an abrupt end.

Event Rules

An Event Rule determines when a process or user violates one or more of the organization's policies and standards.  Event rules would seem to be easy to define, but the devil is in the detail.  For example an organization may have an Oracle 10 database standard.  While at first blush the knowledge of which database product used would seem to be the only knowledge rule needed, and the use of a non-Oracle 10 database product would seem to be knowledge needed to trigger the event rule, in fact it might be much more complex.  For example, suppose there is a waiver to the database standard based on a contractual obligation.  Then, the event has not occurred because two metrics are required, 1) that "a non-standard database is used" and 2) that "there is no waiver".  You can see that if this most basic event rule requires more than one knowledge rule to cause the event, then most (business or organizational) Event Rules will require many more.

Value Rules
A Value Rule determines the value of  the consequence of a violation of the policy or standard.  While the Knowledge and Event Rules are the "If" clause  of the "If-Then-Else" branching logic statement in programming and in business processes, the Value Rule is the "then" clause.  The Value Rule determines the value of the various alternatives based on the cause of the Event, that is, the combination of Knowledge Rule values.  For example, if the police officer pull a teenager over for not using a direction indicator (turn signal) on her car when changing lanes on an empty Interstate highway, the officer may give her a warning, while, if the Interstate is busy, the officer may give her a ticket.  Again, if it is the driver's first offense, the officer may determine that a warning is sufficient, while if the driver has had one or more warnings, the officer may write a ticket.

Summary
If an organization can formally decompose their policies and standards into these three types of Business Rules, and can insert them into a central Rule Repository, which is part of the AEAR, then organization can be both much more agile and, at the same time, in control of changes in the policies and standards because the Enterprise Architect will be able to model the effects of any changes.

Queensland’s Strategic Cropping Land

I have been critical about the farming lobby’s reaction to the Murray-Darling Basin Plan, and I have also been very critical about the value of food security, especially when used as a justification for agricultural subsidies.

My general belief is that farmers should be treated like any other business and face risks from their investment decisions.  Because this belief I strongly support Queensland’s new Strategic Cropping Land Policy. 

The policy under development gives farmers a chance to opt out of mining and gas production on their land. Currently land owners must allow mineral and gas exploration and development on their land. The mining industry has legislative power behind it to explore for, and mine, the States mineral resources (have a look at your title deed and you will note that even freehold land owners don’t own the minerals under their land).

This means that miners do not need to buy any property rights from existing land owners to conduct activities on privately owned land. They do however need to provide some compensation for disruption to activities (as prescribed under the relevant acts).

In the greatest of ironies, agricultural policies in this country have protected farmers from their own business decisions (eg. subsidising water supplies, making drought and flood payments - I argue these events are part of the natural weather cycle and should be anticipated), yet have not protected farmers from external threats to from mining.

It took a while for the food security lobby to realise that the food production of the country rests in the land, soil and water, not in the individual businesses of farmers. If a farm business fails, the productive capacity remains for the next buyer of the property. But if land, soil and water is irreversibly damaged, then potential food production capacity is destroyed.

With these bizarre policies in place it is possible to have the situation where a farmer is receiving drought relief payments on the one hand to save his business, while the government is supporting the demise of his ability to farm on the other hand by allowing coal seam gas wells to be peppered across his fields.

In the Darling Downs the preservation of the water quality in underground aquifers is especially important. These aquifers are a significant source of water for agriculture and there is a reasonable probability that drilling through this aquifer many thousands of times to reach the deeper coal seam will contaminate the water. And unlike a river system which flushes water readily, underground aquifers may take hundreds of years to recover (or water users will need to treat the now contaminated water before applying to crops).

The irreversibility of mining and coal seam gas impacts is one of the key reasons that farmers should be given some ability to opt out of such activities on (or even near in some cases) their land.

The outcomes from this type of policy should satisfy a broad range of interests.

1. Land use conflicts are more easily resolved by given some powers back to existing land owners.

2. By protecting the land itself those who want food security and local food produce benefit.

3. Those who want ‘agricultural open space’ benefit (people actually like knowing there are farming communities and driving through the country).

4. Farmers who want to be free to run their own business, protected from irreversible land damage benefit.

5. Those who want mining can do so if the impacts on surrounding land owners are sufficiently low.

Of course there will be problems to overcome during implementation, but in principle the policy appears sound. An indeed, the minerals and gas remain in the ground should future circumstances require their extraction.