WP047: A Proposed Trajectory of Validation Experiments for Computational Emulation Models of Organizations
Abstract/Contents
- Abstract
- Validation of complex simulation models, with multiple inputs and feedback loops, is a challenging, multifaceted problem. It has received considerable attention in the Computational Organizational Science (COS) literature. The COS literature typically calls for extensive validation and discusses some guidelines for when certain techniques are appropriate. However, reports of rigorous external model validations are limited. In this paper, we use an organizational design and analysis tool, called the Virtual Team Alliance (VTA), to illustrate that one needs to perform a series of validation steps in a predefined sequence to accomplish a comprehensive, credible validation of a computational organizational simulation model. The purpose of the VTA model is to provide managers with a tool that they can use to test "virtual prototypes" of project organizations and predict outcomes prior to actual implementation-it can therefore be described as an "emulation" system. The ultimate external validation of VTA is whether or not it is useful to managers for this purpose. VT A is useful if the model forecasts problems that will occur without organizational change, managers redesign the organization based on the model's problem predictions and suggested remediations, and the organizational risks, which the model predicted are thereby reduced. To reach this kind of ultimate external validation of usefulness, VTA has to go through a "trajectory" of different validation methods. The primary contribution of this paper is the development of an innovative validation trajectory strategy for complex computational emulation models. We present a validation trajectory that includes (1) computational synthetic experiments, (2) retrospective validation and comparison with manager's "what-if' predictions, (3) contemporaneous validation, and (4) prospective validation with intervention. We discuss in some detail how we applied VTA to two portions of an ongoing aerospace project. The VTA model made predictions about severe bottlenecks and potential quality problems for one sub-team within the two project teams but no problems for the other sub-team. These predictions were subsequently confirmed. The results of our experiments agreed with those of extant, qualitative organizational theory.
Description
| Type of resource | text |
|---|---|
| Date created | March 1998 |
Creators/Contributors
| Author | Thomsen, Jan |
|---|---|
| Author | Levitt, Raymond E. |
| Author | Kunz, John |
| Author | Nass, Clifford I. |
Subjects
| Subject | CIFE |
|---|---|
| Subject | Center for Integrated Facility Engineering |
| Subject | Stanford University |
| Subject | Computational Organizational Simulation Models |
| Subject | Contingency Theory |
| Subject | Information Processing |
| Subject | Intervention |
| Subject | Organizational Design |
| Subject | Validation |
| Genre | Technical report |
Bibliographic information
Access conditions
- Use and reproduction
- User agrees that, where applicable, content will not be used to identify or to otherwise infringe the privacy or confidentiality rights of individuals. Content distributed via the Stanford Digital Repository may be subject to additional license and use restrictions applied by the depositor.
Preferred citation
- Preferred Citation
- Thomsen, Jan and Levitt, Raymond E. and Kunz, John and Nass, Clifford I.. (1998). WP047: A Proposed Trajectory of Validation Experiments for Computational Emulation Models of Organizations. Stanford Digital Repository. Available at: http://purl.stanford.edu/tr154qn4424
Collection
CIFE Publications
Contact information
- Contact
- cife-email@stanford.edu
Loading usage metrics...