Your Simulation Questions Answered

Mike Allen's picture

So, you’re thinking of using simulation within your organization. Is simulation worthwhile? What are the pros & cons? Should you recruit an in-house team or outsource? What skills should your team possess? Which software should you purchase? How should you conduct simulation studies? Are there any other gotchas?

Good questions, all. In this article, I’ll attempt to cover each as briefly as possible. If you would like more detailed explanations on any point, feel free to call me on +1 (313) 451-4001 and I’ll do my best to assist.

Is Simulation Worthwhile?

Like a lot of these answers, this will depend upon your circumstances.

If your production, manufacturing or distribution processes (or the processes that you design or supply to your customers) are very simple and well understood, and you can predict with a high degree of certainty how they respond to change, then it’s unlikely that simulation will provide you with a great deal of added value. Similarly, if you only plan to process a small number of low-value items, then simulation may prove uneconomic.

Simulation starts to become a valuable resource when uncertainty, complexity and risk start to enter the equation, and becomes more valuable the more of each you have. If you have a high volume, high product value, complex, mission-critical system whose behavior you cannot reliably predict, or if you are supplying such a system to a customer, then using simulation should be a no-brainer.

Of course, you can choose to either simulate and then implement, or just implement. If your process design is bad, and you discover that fact by simulating it, then your costs will be relatively slight. If you skip the simulation, and implement your bad design, your costs will typically be astronomically higher.

What Other Benefits Can I Realize From Utilizing Simulation?

There are many benefits:

  • Computer simulation models execute many times faster than real time, so you can cover far more ground with simulation models than you can by experimenting with your real-life process. Depending upon the complexity of the model, you can simulate years of production in a matter of minutes or hours.
  • Modeling a process requires that it be studied in detail, often highlighting its more obvious problems before the simulation even exists. Many of the benefits of the simulation come from this phase of a simulation study alone.
  • Simulation models typically produce a number of informative metrics that the real-life system does not, giving you far greater insight into the performance of that system.
  • Every simulation study I have ever been involved with has highlighted at least one significant, but unanticipated, problem with the design of the planned process.
  • Simulation models can test your designs thoroughly, under many different conditions and scenarios—far more thoroughly than any human process designer. Furthermore, it’s often possible to be far more radical with your designs, given you have a simulation safety net.
  • Once you have discovered a problem, you can model the alternate solutions, simulate them all, and identify the best objectively. There's no need to fall back upon guesswork or gut-feel.
  • You can use the many different simulation experimentation tools to tune your process' operating parameters for peak performance.
  • Most simulation tools generate intuitive animations—some in true-scale 3D—that allow you to use the simulation as a focus for your process team's creativity. Everyone can see the problem(s) and make suggestions or even make course corrections. You can then simulate these and identify their merits—politics never comes into it!
  • These same animations make great sales tools if you need to sell your design to others. Not only do they bring a design to life, they show its true operation, warts and all.

Is There a Downside To Utilizing Simulation?

Of course, there's no such thing as a free lunch, so there are disadvantages too:

  • A professionally-executed simulation study will cost money—but you will typically get a very rapid payback.
  • Conducting a simulation model takes time—anywhere between a day to a few months, depending upon the complexity of the process you're modeling and your objectives.
  • Unfortunately, this is often a critical path activity between completion of the pre-simulation design and its procurement and implementation. In my experience, project managers are loathe to make sufficient time for a full simulation study, but that time is an excellent investment. This is akin to the old adage: marry in haste, repent at leisure.
  • You must have good data to hand. If you don't have good data, then you can perform sensitivity analysis across a range of values to gain better insight as to whether the accuracy of that data is significant or not. However, that can take more time to accomplish (paralysis through analysis) and will not necessarily lessen the uncertainty in the results. However, if you don't have good data, then you have far bigger problems designing a process than simply creating an accurate simulation model.
  • Occasionally, it's necessary to break the news to the owner of a system that they have an ugly baby. But they're going to find out sooner or later anyway.

Should I Recruit an In-House Simulation Team or Outsource?

The deciding factor is going to depend upon the demand for simulation work, and the fluctuations in that demand.

Ideally, you will want to do your simulations in-house. An in-house team is likely to be:

  • More cost-effective.
  • Flexible to changing priorities.
  • Familiar with your processes, products and services.
  • Proactive in seeking and eliminating wasteful practices.

At the same time, external resources are likely to be:

  • More experienced, seasoned practitioners.
  • Flexible to changing demand.
  • Familiar with new techniques and best practices.
  • More likely to offer a range of software and services.

I strongly recommend that all of your in-house simulation staff work on simulation full-time. If you do not have sufficient demand to justify at least one permanent simulation employee, then you ought to outsource all of your simulation work.

If you have sufficient, consistent demand, then you’re likely going to need an in-house team. If your demand fluctuates, then hire a large enough in-house simulation team to handle the troughs, and hire external resources as needed to take up the slack.

What Skills Should A Simulation Practitioner Possess?

Simulation practitioners must be multi-skilled:

  • Modeling skills. An essential skill is the ability to correctly define a system, determining which components are part of that system and which are a part of its environment, and how those components interact with each other. Also included is the ability to define objectives for system studies, resulting in the specification of models that will satisfy those objectives, and for validating the resulting models.
  • Statistical analysis skills. This is essential for specifying, collecting and analyzing data used as inputs to the simulation, as well as for analyzing the output of the simulation. This skill is also required for designing experiments to be conducted upon simulation models, and for specifying configuration options for each simulation.
  • Software engineering skills. The development and verification/testing of a simulation model has a lot in common with software engineering. The model must be architected, specified, programmed and tested in an accurate, reliable and timely manner. (Many software vendors like to dumb down these skills, by claiming how easy-to-use their products are. Still, there isn’t a single software product that does everything you’ll need out-of-the-box, and—because of the complexity often inherent in the systems being studied—you will always need some custom logic in your simulation. Even when the software does provide everything you need, you still need to verify that the model is bug-free and an accurate representation of the real-life system.) In any case, the practitioner needs to be fully familiar with the simulation software that you will utilize.
  • Communications skills. At the beginning of a project, it is essential that the simulation practitioner understands the nature of the problem and the reasons why a simulation is to be performed. These are not always obvious—even to the people commissioning the simulation study. Furthermore, the practitioner needs to interview a number of process experts to understand how the process operates, in order to tease out the relevant information. As the simulation study progresses, the practitioner needs to be able to explain the simulation, solicit and address criticisms of it, and, finally, present and explain its results and recommendations. Also necessary are succinct and accurate written documents, such as specifications and reports.
  • Miscellaneous other skills. Depending upon your industry, your simulation practitioners may also need to be skilled in understanding material handling system equipment, controls engineering, the nature of the processes you specify, support or supply, etc.

If you have a large team, you can have individuals who are deficient in one or, maybe, two of these skills provided that other team members can fill in the gaps. If you have a small team, then you have little option but to hire high caliber individuals who cover all the bases. If you’re outsourcing, your simulation vendor must be able to demonstrate their proficiency.

Note that some of these skills are technical and can be acquired through training, while some can only be honed from long experience on the job.

It should also be noted that you are taking a risk if you do not personally possess these skills yet attempt to hire someone who has, or who claims to have, them. How can you evaluate their skill set? In this case, I recommend that you seek the assistance of an acknowledged, trusted expert to grill your potential hires, particularly if you are setting up a brand new simulation team.

One common mistake I see is for organizations to assume that anyone with, say, an industrial engineering, or operations research, background can simply step into the role: this is most definitely not the case.

Which Simulation Software Products Should You Use?

There are many different simulation tools available, including (alphabetically): AnyLogic, Arena, AutoMod, Demo3D, Enterprise Dynamics, ExtendSim, FlexSim, Plant Simulation, ProModel, Simio, Simul8, SLX and Witness to name a few. They all have different capabilities, features, graphics, paradigms, application areas, ease of use and—obviously—pricing. They all have their strengths and weaknesses, and, clearly, some are better than others.

I don't endorse all of these products and I'm certainly not going to recommend any without knowing your circumstances and requirements.

Categorizing these products is not trivial and I will not attempt to do so (otherwise I shall be fending off phone calls from disgruntled software sales staff for the foreseeable future).

While it’s true that just about every product can handle just about every simulation task, it’s also true that you can mow a football field with a pair of nail clippers—some tools are more suited to a particular task than others.

It should be also pointed out that your simulation needs may change during the course of a single project. For instance, in the early stages of a project, you might be more concerned with capacity planning, determining line rates and buffer sizes. At this point, a simulation product with high-level, abstract modeling capabilities will likely meet your needs. By the time you have a detailed system design, you might require a tool that accurately represents the detailed sequence-of-operations in your system, to ensure that it will meet the required production rates, and using an abstract modeling tool will not be your best option.

Your needs should dictate your choice of software, rather than your software dictating what you can do.

Again, consulting with an independent expert who has used a number of different products and who understands your needs will probably be more useful to you than inviting a series of salespeople in to demonstrate their products (although that is always a useful first step), each telling you how their software is the best fit for your needs.

Obviously, if you outsource your simulation requirements, you gain more flexibility and do not have to invest in one or two expensive software products.

How Should You Conduct Simulation Studies?

Every simulation practitioner should have a methodology for conducting their studies, to ensure the quality of the results. While there is no one methodology that every practitioner subscribes to, there are some commonly-agreed steps that everyone should adopt:

  • State the problem.
  • Specify the objectives.
  • Develop and specify a conceptual model to satisfy the objectives.
  • Approve the specification of the conceptual model.
  • Develop the model, by translating the conceptual model into your simulation software.
  • Verify the model, to ensure that it executes bug-free and reports accurate statistics.
  • Validate the model, to ensure that it is an accurate representation of the real-life system.
  • Approve the model.
  • Experiment with the model, addressing the objectives and forming conclusions and recommendations.
  • Present the findings and issue a final report.

If you have an in-house team, you should formulate a detailed, standard methodology along these lines that you require each simulation study to follow. You should also specify this same methodology when hiring simulation service vendors (although it’s recommended that—in the spirit of continuous improvement—you also solicit their comments and feedback) when outsourcing simulation work.

If you only outsource work, remember to ask potential vendors for a copy of their methodology and ensure that—at the very least—they specify, verify and validate their models. (To understandard why, see my article on simulation quality control.)

Is There Anything Else I Need To Consider?

I strongly recommend that you have an experienced simulation professional conduct, or at least supervise, your first simulation project. The single biggest problem I face professionally is overcoming the skepticism of people who have been burned by a poorly-executed past simulation exercise. If you choose to not utilize an experienced simulation professional and your first project goes badly, you may be tempted to give-up on simulation, and that would be a huge mistake.

Perhaps the most important consideration of all: if you do not have support for undertaking simulation work at an executive level, then it can become extremely difficult to conduct effective simulation studies. You simply must have a simulation champion at a high-level within your organization. If you do not, many other team members will not regard a simulation study, or their involvement with it, as a high enough priority. (Imagine you're the manager of your simulation team trying to convince a project manager that your simulation study is a critical path activity without the benefit of someone senior to the project manager backing you up.)

What's Your View?

Hopefully, I addressed some of your questions and concerns, and provided some food for thought. However, I welcome your comments and suggestions, so feel free to pass on your thoughts below.

If you would like to discuss any of the information presented here, feel free to give me a call on +1 (313) 451-4001 or send me an email.

Mike Allen

President, Hindsight Consulting, Inc.