A Comparison Study of Different Optimizing Criteria and Confounding Patterns For Multi-Level Binary Replacement and Other Designs Used in Computer Experiments

Abstract

We have constructed four different types of designs for computer experiments. Thedesign types are based on latin hypercube sampling (LHS), orthogonal arrays (OA), ran-dom sampling and the recently proposed multi-level binary replacement (MBR) design.For each type of design we have attempted to find the best possible design out of acertain number of constructed designs using three different optimizing criteria: the alias sum of square criterion (ASSC), the L-criterion and a modified A-criterion. The chosen design has then been tested by fitting an approximate model and calculating maximum error (MAX) and root mean squared error (RMSE) values. We observed that out of the three criteria applied the ASSC performed the best.In addition to comparing criteria for optimizing the design choice, we have alsoconstructed non-optimized designs for comparing the different design types and thedifferent ways of constructing MBR designs. In this setting we observed that OA designsperformed well in general, whereas the MBR designs performed well when restricted toa small number of factors.