Simulation Quality Control

How do you know if the results of a simulation analysis are accurate and reliable?

This is an important and fundamental question. If a simulation produces invalid results, yet the client accepts and acts upon them, then they’re heading into a world of pain. Just as bad is the situation in which the results are valid, but are distrusted by the client and so are disregarded. In both cases, the outcomes can be catastrophic.

A Checkbox Simulation Horror Story

A checkbox simulation is a simulation study that is done purely to comply with a company policy that new facility designs must be simulated—but typically without any criteria specifying the quality of that simulation. What follows is a fictitious story—yet one that’s all-to-common in my experience—about the limitations of such simulation studies, and the accompanying race-to-the-bottom by consulting firms competing to perform them. I’m glad to point out that Hindsight is not one of them…