Evaluating a Formal KBS Specification Language

In the last few years, formal modeling languages have begun to play an increasingly important role in the knowledge acquisition community. A testimony to this is the steady stream of proposals for such formal languages for knowledge-based system (KBS) modeling. These modeling languages differ from both the high-level informal modeling languages-for example, those used in CommonKADS--and from directly executable languages.Various authors have argued the advantages of such formal modeling languages: They reduce the vagueness and ambiguity of informal descriptions, enable validation of completeness and consistency through formal proofs, and bridge the gap between the informal model and the system design.However, these advantages come at a price. As the software engineering community well knows, formal modeling languages suffer from problems that severely limit their usefulness. They are often not expressive enough to handle real-world applications, and they are usually complex and hard to read. Moreover, constructing a formal model is difficult, error prone, and expensive.In this article, we describe a study that investigated the usability of a formal KBS modeling language, (ML). Fully developed by 1990, this language aims specifically at formalizing the CommonKADS expertise model. At the time we conducted our study, the language definition had become stable. In addition, the language, along with a set of support tools, had already been applied in several cases.To analyze (ML) usability, we first designed a set of evaluation criteria. Then we performed a small case study, constructing an expertise model in (ML), to test and refine this criteria. Subsequently, we used (ML) to construct a second model, which formed the basis for our language evaluation. Altogether, we performed three case studies, which we used to score (ML) with our evaluation criteria.