Simulation and modeling - Old Questions

11.  Why do we need the analysis of simulation output? How do you use simulation run statistics in output analysis? Explain.

5 marks | Asked in 2073

Output analysis is the analysis of data generated by a simulation run to predict system performance or compare the performance of two or more system designs.
We perform the analysis of simulation output because output data from a simulation exhibits random variability when random number generators are used i.e. two different random number streams will produce two sets of output which (probably) will differ. So, statistical techniques must be used to analyze the results.  It provides the main value-added of the simulation enterprise by trying to understand system behavior and generate predictions for it. It also helps to test different ideas, to learn about the system behavior in new situation, to learn about simulation model and the corresponding simulation system.

Simulation run statistics
In the estimation method, it is assumed that the observations are mutually independent and the distribution from which they are draws is stationary. Unfortunately many statistics of interest in simulation do not meet these conditions. An example of such case is queuing system. Correlation is necessary to analyze such scenario.  In such cases, simulation run statistics method is used.

Example:                                                    

Consider a system with Kendall’s notation M/M/1/FIFO (i.e. a single server system in which the inter-arrival time is distributed exponentially ;and service time has an exponential and queue discipline is FIFO) and the objective is to measure the mean waiting time.

In simulation run approach, the mean waiting time is estimated by accumulating the waiting time of n successive entities and then it is divided by n. This measures the sample mean such that:

Whenever a waiting line forms, the waiting time of each entity on the line clearly depends upon the waiting time of its predecessors. Such series of data in which one value affect other values is said to be autocorrelated. The sample mean of autocorrelated data can be shown to approximate a normal distribution as the sample size increases.

A simulation run is started with the system in some initial state, frequently the idle state, in which no service is being given and no entities are waiting. The early arrivals then have a more than normal probability of obtaining service quickly, so a sample mean that includes the early arrivals will be biased.