Detecting the Knowledge Frontier: An Error Predicting Knowledge Based System

Knowledge Based Systems (KBS) have long wrestled with the\ problem of incomplete knowledge that occasionally causes them to make ridiculous conclusions. Knowledge engineers have searched for methodologies that allow for less brittle systems. Additionally, KBS systems for general knowledge have been developed to try and build background information that a system can fall back on when they cannot find a conclusion in their specific domain. However, it is next to impossible to include all the required knowledge to completely eradicate the inherent brittleness of these systems. This paper presents a method for predicting when a case being presented to the KBS is outside its current knowledge. When the system notices such a case it provides a warning allowing the user to investigate the case further. This preliminary study of the system has been tested using a simulated expert with randomly generated data sets. It shows that this system has great potential for predicting almost every error and rarely issuing warnings for correct conclusions. Such a system could significantly reduce the knowledge acquisition task for an expert. These results clearly show the potential of such a system and that further investigation with recognised data sets and real user tests should be performed.