Kobi Gal, Ann Nowé, Grzegorz J. Nalepa, Roy Fairstein, Roxana Rădulescu (eds.)
Frontiers in Artificial Intelligence and Applications (ECAI 2023) 372
2023
Symbolic knowledge-extraction (SKE) techniques are becoming of key importance for AI applications since they enable the explanation of opaque black-box predictors, enhancing trust and transparency. Among all the available SKE techniques, the best option for the case at hand should be selected. However, an automatic comparison between different options can be performed only if an adequate metric - such as a scoring function resuming all the interesting features of the extractors - is provided. The definition of evaluation metrics for symbolic knowledge extractors is currently neglected in the literature. Accordingly, in this paper we introduce the FiRe score metric to assess the quality of a symbolic knowledge-extraction procedure, taking into account both its predictive performance and the readability of the extracted knowledge. It is compared to another existing scoring metric and a rigorous mathematical formulation is provided along with several practical examples to highlight its effectiveness to the end of being exploited inside automatic hyper-parameter tuning procedures.
origin event
funding project
TAILOR — Foundations of Trustworthy AI – Integrating Reasoning, Learning and Optimization
(01/09/2020–31/08/2024)
works as
reference publication for talk
Symbolic Knowledge-Extraction Evaluation Metrics: The FiRe Score (ECAI-2023, 04/10/2023) — Federico Sabbatini (Federico Sabbatini, Roberta Calegari)