Evaluation of Self-Assessment Tools - A Case Study

Abstract

While self-assessment tools are widely used, little research has been conducted regarding the evaluation of such tools. In addition, there is a widespread misconception regarding self-assessment tools, where they are considered and used as if they were measuring instruments, while they in fact provide nothing close to valid measures. In this thesis we provide a conceptualization of self-assessment tools as facilitators of a process where its users build an image of the organization. This image then forms the basis for changing the organization. The Psychiatric Health Services Division at IHT is experiencing major changes in their context and are therefore open to introducing tools that can aid them in the changes this requires them to make. Specifically, they have been looking at the LESAT, a self-assessment tool developed at MIT. Such tools, taken from another context, should be evaluated before adopting them to an organization. We propose three criteria that serve as a framework for evaluating self-assessment tools. To help the division in their evaluation, we have conducted a case study, evaluating the LESAT for use in their organization. The result of the evaluation was that the LESAT is inappropriate for use in the division, and its design is found to focus attention towards generating numerical scores rather than encourage reflection and help users improve on their image of their organization.