There seems to be general agreement amongst those involved in KBS research that in order to be useful, a system must be able to explain its reasoning to a user. This paper reviews the development of explanation facilities in knowledge-based systems. It differentiates between explanation as a problem-solving process, and that which explains a reasoning process. This review concentrates on the latter, identifying and giving examples of three categories of reasoning explanation.
We then look at user requirements for explanation. What makes an explanation useful depends on the expectations of a user, which in turn depends on such issues as user background and system context. Several techniques are examined that have been applied to the problem of producing explanations that are appropriately structured and conveyed.
Finally, we discuss some of the work that has been done in describing theories of human discourse and explanation, and some issues that will become increasingly important for future explanation systems.
An extensive annotated bibliography is provided.