Simulation and Modelling - Old Questions
8. Explain the three step approach of validation of models in simulation
Naylor and Finger proposed three step approach for validation process. These steps are as follows:
1. Face Validity
2. Validate Model Assumptions
3. Input - Output Transformation
1. Face validity
- A model should appear reasonable on its face to model users and to those who knows about the real system that is being simulated.
- A model should be designed with high degree of realism regarding system structure and behaviour through reliable data.
- The potential users should also be involved in the validation process to aid in identification of model deficiencies and optimizing those deficiencies to produce better model. This process is termed as structural walkthrough.
- Sensitivity analysis is also used for face validity of the model. It analyses the effect on output when there is change in input parameters. Sensitivity analysis is done through appropriate statistical techniques.
2. Model Assumptions
Model assumptions fall into two categories: structural assumptions and data assumptions.
- Structural assumptions involve questions of how the system operates and usually involve simplification and abstractions of reality.
- Data assumptions should be based on the collection of reliable data and correct statistical analysis of the data. It deals with such questions as what kind of input data model is? What are the parameter values to the input data model?
3. Input-Output Transformations:
- It involves validating whether the model can predict the future behaviour of the real system when the model input data match the real inputs and when a policy implemented in the model is implemented at some point in the system.
- In this validation phase, the model accepts values of the input parameters and transforms them into output measures of performance.
- The modeler uses the historical data reserved for validation purposes only.
- For the complete input-output validation, at least one set of input conditions should be collected from the system data so as to compare to model prediction.