Vol.1, Issue 4, Jul - Aug 2003

Return to the Front Page

Systems, Simulation and Data Mining
(Part 1 of a 3 Part Series)

Dorian Pyle
Data Miners

System simulation is a powerful tool. Data mining is also a powerful tool. Used separately they each will help any company make better and more informed decisions. However, used together they form a far more powerful combination for driving value through justified knowledge and insight than when used separately. Why? Or, perhaps more importantly: How?

This is the first part of a three-part discussion of where the value of using these tools in combination comes from, and how to find it for yourself. The author of this article is also the author of the recently published book Business Modeling and Data Mining (Morgan Kaufmann Publishers) which discusses all of these issues in far more depth and detail. This first part briefly discusses why this powerful combination of tools works. The next two parts look at how to use the tools to leverage value in a business setting.

Dorian Pyle is a recognized authority on capturing business value from data using analytics techniques, and has successfully combined those techniques with simulation on a number of projects. As a consultant and instructor, he has helped companies generate high revenue and reduce costs in many industries.
Dorian is a consultant for Data Miners, a leading data mining consulting firm based in Boston, MA. His new book, Business Modeling and Data Mining addresses how to identify business opportunities and problems that can be solved using analytical techniques, including simulation. He is the author of Data Preparation for Data Mining, which addresses data quality problems in business analytics, and is considered to be the industry-standard work on this topic.

Contact Dorian at:

dpyle@modelandmine.com
dpyle@data-miners.com
Direct Line: + 978.537.8288
Personal Website: www.modelandmine.com

The real value of systems representation, and more particularly systems simulation, is that it provides a succinct and concise encapsulation of knowledge. It is an active, usable and testable symbolic representation of how some part of the world works. Managers can explicate and exemplify their understanding and test it without the risk of running a real-world prototype. Where management appreciation of the world is accurate and sufficiently complete, a system representation can provide a useful and valuable representation of the problem or opportunity domain. Unfortunately, human appreciation is often all too fallible and flawed, in which case the system representation merely represents a fantasy. When grounded in reality the models are highly valuable - otherwise they can lead decision makers astray. But how can the necessary grounding best be achieved? When data is available, data mining provides the answer.

A powerful tool for knowledge discovery in its own right, data mining is the ideal companion for generating systems representations and simulations. Data mining is an analytical tool that enables a skilled practitioner to discover - in data of course - relevant objects and the actual relationships that exist between those objects. Systems simulation provides a vehicle to represent those objects and relationships and then to add user insight from experience and prior knowledge that is not represented in available data.

Taken together, a system representation informed by data mining and tuned with expert knowledge beats using either tool alone. The results can be dramatic.

In one case a simulation of a business situation had already been created and was being used as a management decision tool. The company is a major materials handling vendor located in the US. The simulation had been created from a significant management process, based on senior and middle managers' experiences and insights, and was credited with improving corporate responsiveness to client requests by nearly 10% since its inception. (An improvement that management felt well repaid the investment required to create the model.) But the simulations frequently produced results that turned out not to match actual results, so the model was evidently far from perfect.

As a method of better calibrating the model, the team responsible for the simulation model were persuaded to try mining their existing data. It turned out that two crucial relationships in the simulation were mischaracterized - not by much, but the non-linear interactions in which these relationships participated affected the whole model. It also turned out that carefully examining the data revealed that two of the management "levers" relied on by senior management appeared in practice to have no effect whatever on the real-world process that they were intended to manipulate.

Using the insights gleaned from mining the data resulted in the recalibrated model improving responsiveness by a documented 30%. Yet without the system simulation to show where the optimal use and effect of the discoveries in the data could be applied, it is highly doubtful that the results of the mining, in and of themselves, could have been practically applied. It was the combination of these two techniques that returned the value.

The technological and business world is a very complex place - and becoming more so by the day. There is a desperate need for new approaches and tools to support and supplement managerial intuition, experience and other "traditional" methods of making decisions. Nothing in the immediate future will replace human judgment. But that judgment has to be supplemented and supported by the most powerful tools available. Fortunately business system simulation, coupled with data mining and other powerful analytic techniques, provides the same support to decision making that steam did to muscle power at the beginning of the industrial revolution.

The next part in this series looks in more detail at incorporating these tools and techniques into a company's decision-making practices.


46 Centerra Parkway, Suite 200, Lebanon, NH 03766-1487 Phone 603-643-9636 / Fax 603-643-9502