Applied Information Economics (AIE): Not just calculate, scientifically MEASURE IT

Introduction

The economic benefit analysis of an Enterprise Architecture Integration program is directly related to the success of an Enterprise Architecture program. However, the benefits of Enterprise Architecture cannot be measured in a meaningful way. Therefore, the result is that Enterprise Architecture is not as widely adopted, as it otherwise would be. This article explains how Applied Information Economics (AIE), a revolutionary methodology invented by Douglus Hubbard of Hubbard Decision Research, fills in those unanswered gaps that act as a dangerous trap for enterprise investment decision makers. This article explains why AIE is so important in helping organizations realize the actual benefits of an Enterprise Architecture Program and all other subsequent projects executed under the hood of this Enterprise Architecture program. 

Applied Information Economics: A brief overview

Applied Information Economics (AIE) is the practical application of mathematical and scientific methods to the enterprise decision and valuation process.  AIE distinguishes itself from all other decision making methodologies by its synthesis of a variety of scientific and mathematical fields which makes possible everything measurable. AIE uses advanced methods from decision theory, portfolio optimization methods, operations research, statistics, and many other areas to solve even the most difficult IT valuation issues. It also uses the same methods that actuaries, financial analysts or statisticians use.  AIE can measure any IT benefit or risk including faster communication, better quality data, or customer satisfaction.

The tools of economics, financial theory, and statistics are all major contributors to AIE. In addition to these more familiar fields, AIE includes Decision Theory - the formulation of decisions into a mathematical framework - and Information Theory - the mathematical modeling of the transmission and reception of information. It is important to emphasize, however, that even though AIE is a theoretically well-founded set of techniques, it is a very practical approach. Every proper application of AIE keeps the bottom line squarely in mind. All output from the AIE project is in support of specific practical business objectives.

Figure 1: AIE Foundation Blocks. Source: Hubbard Decision Research. Used with permission of Douglas W. Hubbard.

AIE significantly deviates from the traditional cost benefit analysis procedures which rarely contain real measurements. Paradigm shifts in AIE include following:

• Everything is measurable
• The purpose of measurement is to provide information to make better future decisions not merely to justify past decisions
• Using range estimates for costs and benefits to estimate the value of IT is better than using averages or best guesses as estimates
• The value of information needed to make decisions can be computed
• Uncertainty and Risk can be quantified
• Scientific methods of measurement are practical for IT investment.

AIE thus eliminates all the executive hypes and pseudo measurement illusions with the introduction of real scientific probabilistic measurement.

AIE Steps

AIE suggests that instead of choosing a single point value of any set of variables, rather realistic results could be achieved by ascribing the probabilities on them. For this reason, AIE derives the probability distribution graph of the net benefits of a particular type of investment.

Figure 2: Example of a Probability Distribution for the ROI of an IT Investment. Source: Hubbard Decision Research. Used with permission of Douglas W. Hubbard

AIE uses “Monte Carlo” simulation to develop a graph of the likelihood of each possible net benefit. AIE further builds on typical Monte Carlo methods by the use of calibrated estimates and information value calculations.

A “calibrated estimate” is a subjective probability assessment from a person who has been trained to be neither over- nor under- confident in their probabilities.  Decades of research in decision psychology shows that training can compensate for an expert’s tendency to be statistically overconfident in their use of subjective probabilities.  An overconfident person may say they are 90% confident in a series of predictions where, historically, they were right much less often than 90% of the time.  A well-calibrated person, on the other hand, will be right 90% of the time if they say they are 90% confident, 80% of the time if they are 80% confident, and so on.

Calibrated probability assessments allow the decision maker to model the current state of uncertainty about a decision in a way that is neither over nor under confident.  Once this is accomplished, AIE uses standard information-value calculations to compute the value of further measurements for each uncertain variable in the model.  This allows the decision maker to focus on measurements that matter the most.  Finally, once uncertainty is removed to the point where further measurement is not economically justifiable, the decision maker makes a risk vs. return decision.

IT Investment as an Investment Portfolio: AIE Risk Return Analysis

AIE Uses the method of Modern Portfolio Theory and treats the set of IT Investment as another investment portfolio. Each Investment is analyzed on a Risk Return basis with regard to its contribution to the portfolio.

By using techniques from Modern Portfolio Theory, AIE determines whether the uncertainties inherent in a given IT investment decision are acceptable given Risk return profile of the organization. This figure 3 summarizes the Risk Return Analysis (RRA) steps performed by AIE:

Figure 3: Risk Return Analysis Steps

The last step compares the investment to the “investment boundary” of an organization.  This is a chart that shows how much risk is acceptable for a given return for a particular investment size.  Here is an example graph drawn against an IT investment of $2M-$3M plotted against a client’s investment boundary. In this case, risk is shown as a chance of getting a negative internal rate of return (IRR) from the investment (computed from the Monte Carlo simulations).  The objective of an investment is to plot below the investment boundary where the risk is more than acceptable given the return.  Such a graph would sometimes show that a lower return investment is preferred to a higher return investment if the risk of the first is much lower.  This Graph suggests that the 27% IRR investment is actually preferred to the 83% IRR investment.

Figure 4: Example Risk Return Profile. Source: Hubbard Decision Research. Used with permission of Douglas W. Hubbard

The net result of this process is to present the uncertain value and risks of any investment (like EA) in a way that would be consistent with more sophisticated actuarial and statistical analysis.  Not only does this quantify risk in a meaningful way, but it optimizes the measurement process itself by focusing on the highest payoff measurements.

The lesson learnt for the Enterprise is that measurement is the key for any investment decision. MEASURE is a 7 step iterative analysis procedure that refines and simplifies any IT decision making process. Let me explain how to overcome this MEASUREment challenge.

MEASURE: A simplified capsule for IT Decision makers

Measurement is most important for any correct IT decision. The Purdue Enterprise Reference Architecture (PERA) Master Plan provides an excellent means of economic justification for any decision, classified into three important categories:

1. Economic: suitable for projects with strictly economic benefits
2. Analytic: appropriate for projects with both economic and non-economic benefits
3. Strategic: appropriate for systems with strategic advantages

However, “a strategic justification accompanying an economic one may assure justification of a project that might well decide whether the company will become a competitive force in the market or disappear from it. It may be that a project that yields little return is justified from a strategic viewpoint because it is a prerequisite for other profitable follow on projects” [PERA]. Therefore, a strategic viewpoint alone should not drive any EA investment plan under any circumstance. Projects ought to be spawned based on the outcome of the economic benefit analysis process and by prioritizing the payback potential.

MEASURE is a simple 7 step analysis procedure introduced by the author. These seven steps are as follow:

• M = Metrics. In our step 1, it is highly recommended that one makes a consolidated list of appropriate metrics, which includes but is not limited to models, charts and graphs
• E = Evaluate. In this step 2, Metrics are generated.
• A = Analysis. In this step 3, actual interpretation of these evaluations is done.
• S = Select. This is our Step 4 wherein different interested areas of the metrics are picked up and highlighted in order to enable some formatting of conclusive points.
• U = Uncover. This is the actual decision making step.
• R = Return. Here, one returns back to the original problem statement with a particular decision.
• E = Evolve. This step embraces the next iteration point.

Figure 5: What to MEASURE?

In our day-to-day business, we follow some of these well-known economic benefit calculation processes such as ABC, ROI, NPV and TCO. On the other hand, Value Analysis, Portfolio Analysis and Risk Analysis techniques are popular analytical approaches. Application Information Economics primarily acts as a strategic decision enabler, showing us the direction alongside strategic, economic and analytical triangle.

This below-mentioned table summarizes several important observations against traditional CBA versus AIE:

Table 2: Applied Information Economics (AIE) versus traditional CBA

While we have already introduced the reader to the MEASURE magic, it is important to know why enterprise is still far away from the benefit of this MEASURE approach. The main reason behind this impinges upon the spreading of Enterprise Architecture and so the benefits of EA. In our Enterprise, the presence of these antipatterns prevents the realization of the art of enterprise architecture.  In a similar way, so does enterprise benefit analysis and finding the right answer for the fundamental question of “Why EA?”. Therefore, it is important to briefly become acquainted with these obstacles, which are weeding out practices in the art of the EA.

Conclusion

It is a common inversion symptom across the industry that industry prioritises the analysis sequence as follows: Cost, Benefits, Utilization and Chance of Cancellation. AIE unravels this inversion riddle with the recommendation that measurement sequences should be other way around. This article justifies why this AIE recommended measurement sequence is so important and should be started with Risk-Return profile. This article recommends, “To make the best investment decisions, IT management must learn how to compute the value of information, and start modeling utilization and cancellation in cost-benefit arguments. Once IT Management does, measurements will be properly prioritized and the IT measurement inversion will disappear” [The IT Measurement Inversion by Douglas W Hubbard in CIO Enterprise Magazine dated Apr. 15, 1999].

Terminology Puzzles

About the Author

Copyright Declaration

Figure 1, 2, 4 sources are [HDR]. Used with permission of Douglas W. Hubbard

Acknowledgement

Reference

[HDR]Hubbard Decision Research
[How to Measure Anything] How to Measure Anything: Finding the Value of "Intangibles" in Business (Hardcover)by Douglas W. Hubbard, Published by Wiley (Due on August 3, 2007, Ready to Pre-Order), ISBN-10: 0470110120, ISBN-13: 978-0470110126
The Management of Intangibles: The Organisations Most Valuable Assets (Routledge Advances in Management & Business Studies) by A. Bounfour, published by Routledge, ISBN: 0415224934
Life-Cycle Costing: Using Activity-Based Costing and Monte Carlo Methods to Manage Future Costs and Risks (Hardcover) by Jan Emblemsvåg, published by Wiley (March 14, 2003), ISBN: 0471358851
MDA Radar
Applied Information Economics
How To Measure Anything
The Development Risk Premium of IS Projects And Consequences for IS Management By Douglas W. Hubbard
Risk and Uncertainty in Cost Benefit Analysis
US Government Study on AIE
Federal CIO Council comparison of AIE and balanced scorecard
Environmental Protection Agency AIE study about optimizing desktop replacement
Original CIO Magazine "Value Made Visible" article by Tracy Mayor reprinted at Boeing Center for Technology
Dept. of Homeland Security Cybersecurity article discusses the AIE method
How to profit from technology by Ian Grant
What is Break-Even Analysis?
What is Net Present Value - NPV?
What is Value Based Management?
What is Activity Based Costing?
http://en.wikipedia.org/wiki/Modern_portfolio_theory
http://eagroup.blogspot.com/2007/05/roi-from-enterprise-architecture.html
http://www.gartner.com/DisplayDocument?id=366199
http://ocio.os.doc.gov/ITPolicyandPrograms/Policy___Standards/DEV01_002678
http://blogs.ittoolbox.com/eai/business/archives/soa-benefits-challenges-and-risk-mitigation-8075
Hubbard, Douglas W., Personal Advisory Report, the Advisory Council, July 2007