Vol.1, Issue 5, Sep - Oct 2003
Return to the Front Page

Systems, Simulation and Data Mining, Part 2

Dorian Pyle
Data Miners

All business face two fundamental challenges. First, to stay in business. Once the continued existence of a company over some period is assured, the second challenge is to make the best use of available resources to continue to exist and to create surplus value. (Surplus value only means "value over and above that needed to continue operations".)

From one important perspective of a modeler, companies engage in two types of activities: generally typified as "primary" and "secondary". The primary activities are the core activities a company engages in to generate value. They can be characterized as those required to have the right product in the right place at the right price at the right time and in the right quantity. These are the P3 TQ activities.

Dorian Pyle is a recognized authority on capturing business value from data using analytics techniques, and has successfully combined those techniques with simulation on a number of projects. As a consultant and instructor, he has helped companies generate high revenue and reduce costs in many industries.
Dorian is a consultant for Data Miners, a leading data mining consulting firm based in Boston, MA. His new book, Business Modeling and Data Mining addresses how to identify business opportunities and problems that can be solved using analytical techniques, including simulation. He is the author of Data Preparation for Data Mining, which addresses data quality problems in business analytics, and is considered to be the industry-standard work on this topic.

Contact Dorian at:

dpyle@modelandmine.com
dpyle@data-miners.com
Direct Line: + 978.537.8288
Personal Website: www.modelandmine.com

All other activities are secondary, which are those necessary activities that a company engages in to support the primary activities, but they do not themselves directly further the purpose of the company. (This is equally true when the "product" is actually a service.) Paying the electricity bill, obtaining telephone service, paying employees, hiring new staff and training employees are all examples of secondary activities. It is a characteristic of secondary activities that any deficiency in them has an impact on the primary activities, but not the other way around.

For instance, if the administrative mechanism for paying the electricity bill or for providing telephone service failed, the effects would very quickly be felt in the primary activities. However, failure of primary activities would not necessarily affect these secondary activities. (Naturally, if the primary activities fail long enough, the company is out of business and secondary activities fail too - but that's the first fundamental challenge to a company, not the second.)

Modeling - whether through data mining or through systems modeling - delivers its greatest value, and impact, when directly addressing P3 TQ activities and relationships. The reason? This is where a company gains or loses competitive advantage.

Example: Pricing

Each of the core elements in P3 TQ is equally important. No one element can be singled out in isolation as far as importance to a company is concerned. There is, however, no room in this article to more than introduce some of the issues in a single theme. The reader is invited to extend this example, by analogy, to issues other than pricing.

For more than 30 years, companies have attempted to use techniques of yield management, revenue management, strategic pricing (call it what you will) in order to increase revenue through rationally managed pricing strategies. Airlines were the first to try this approach in the '70s, followed by hospitality industries in the '80s, retailers and consumer products companies in the '90s, and representatives from all industries were on the bandwagon in the '00s. The idea underlying these techniques is to understand how revenues rise and fall with changes in pricing and to use this information to optimally set the right price.

There certainly has been no lack of data: POS (point of sale) systems have generated untold volumes of data, for some companies totaling terabytes per year. There has been no lack of analytic tools: data mining has been a powerful contender for analysis for the last 10 or 15 years, and its predecessor techniques for 10 or 15 years before that. Today, a host of software and supply chain automation vendors offer products related to price management, markdown management, marketing mix analysis, and many other related types of analysis. Trade publications have hailed this application as one of the hottest trends, and one that is poised to explode.

And yet. The Professional Pricing Society's recent survey found that only 9% of potential end users use these tools, and that most managers responsible for pricing continue to use experience-based knowledge and instinct, augmented by analysis, to make their pricing decisions. So it seems that analysis alone, even with the most powerful analytic tools available, does not present a compelling enough case for use of the results to achieve optimal pricing. Analysis alone has not provided a "magic bullet". Why?

As any reader of this column should answer, it's because analysis of past data alone does not provide a description of the system in which product pricing is a crucial element. The future is not a repetition of the past; as a wise head once noted, "history rhymes". In other words, certain important features of the system repeat their major effects, but much else changes. This makes straight "prediction", based purely on past data, a less than perfect tool.

One answer is to model the system. Both executive and managerial insights are important, and so too is discovery in data. One major company responsible for managing retail loyalty programs used its expertise and its data to model its pricing system as a whole. Much more was included than simply what pricing data revealed. After creating a large and complex model, the model was simplified through several iterations until reaching what seemed to be the simplest model that described the situation. The relationships within the model were all calibrated against the data using data mining, both to discover relationships and additional objects that had to be included.

Simply to create and calibrate the model took six months. Testing the model took another two months. In the end, several simulations were run to choose appropriate pricing. Result: a 9% revenue growth without any corresponding rise in capital expenditures (which meant double-digit increases in profit levels).

Summary

Pricing is just one case in which analysis is essential - necessary but not sufficient in and of itself. In the project brief described above, system simulation alone would have reflected only informed management opinion and would not have produced sufficiently detailed or accurate conclusions. Analysis alone has proven (in over 30 years of "field trials") to provide a less than completely satisfactory solution. Taken together, however, the synergy of data mining and systems modeling provides a powerful solution to tough problems, including determining the right price for a product or service. Integrating data analysis and system simulation is not a quick or easy solution, but it offers one of the best methods for dramatically improving corporate P3TQ performance.

46 Centerra Parkway, Suite 200, Lebanon, NH 03766-1487 Phone 603-643-9636 / Fax 603-643-9502