Actively Learning EL Terminologies from Large Language Models

In active learning, a learner attempts to learn from a teacher by posing questions. The questions made by the learner are called membership queries and are answered with ‘yes’ or ‘no’. This kind of query is often studied as part of a communication protocol that also includes equivalence queries. Intuitively, equivalence queries ask whether the idea of the learner about the knowledge of the teacher is correct or not. If not, then the teacher should provide a counterexample showing the difference. Here, we consider the teacher as a large language model (LLM) and study the case in which knowledge is expressed as an EL terminology. Membership queries ask whether concept inclusions are true or not. E.g., “Can algae be considered a subcategory of plant?”. Equivalence queries are simulated by a sample with concept inclusions labelled as positive or negative. We present a non-trivial extension of the ExactLearner tool to extract EL terminologies from LLMs. Given the relevant symbols as input (e.g., algae, plant, etc.), the tool tries to find how these symbols should be logically connected by posing questions to LLMs. To evaluate the approach, we present performance results of the ExactLearner in the task of reconstructing existing EL terminologies.