Hmm. That sounds a bit more familiar. One of the
first things we do in creating a model is to define an RBP: a Reference
Behavior Pattern. The RBP focuses our modeling efforts by forcing us to create
a model that can replicate that RBP "well enough." But how do we reduce the
masses of numbers our client or our boss (another word for client, when we are
working internally) may give us into something we can use with integrity?
That's right: statistics.
How do we know our model replicates the RBP well enough? We have a number of
tests, but statistical tests are certainly one means.
Does that mean we all need advanced degrees in statistics to accompany our
hard-won skills in systems thinking? While I'm sure we'd benefit from such
training, we can bring in a statistician for the harder issues. Simple
calculations and tests suffice for many of our issues.
What's important, truly important, is a healthy respect for the data. I
recently was conversing with someone about a particular issue with an eye
towards creating a model to help us understand and possibly address that issue
more effectively. The other person presented a couple of hypotheses that
sounded very plausible. I went back to the office to make the RBP more precise
and to create an initial model that could replicate that behavior.
When I started looking at hard data, though, the RBP graphs didn't look at all
like the informal sketches we had made. While the presumed behavior was
present, it was swamped by an effect that was several times larger. That led me
to a different model and a different focus in my approach. Had I ignored the
data, I might have created a plausible yet ultimately useless model that could
perhaps have sidetracked others' efforts in this area.
So, no matter whether you need a general understanding of the system you're
working in or answers accurate to 1% or better (there are those of us who
regularly do that!), make sure your your model's results are grounded in the
real world.
Make your systems thinking an exercise in reality programming!
