An Empirical Study on the Robustness of Knowledge Injection Techniques Against Data Degradation

Symbolic knowledge injection (SKI) represents a promising paradigm for bridging symbolic knowledge and sub-symbolic predictors in intelligent autonomous agents. Given the wide availability of SKI methods from the literature, we observe that SKI effectiveness is commonly measured in terms of predictive performance variation – e.g., accuracy improvement – introduced by SKI. However, other aspects such as the injection mechanism’s ability to maintain its performance and generalisation capability, despite encountering unexpected or anomalous input during the training process, are equally relevant. Accordingly, in this paper we propose a new metric to evaluate the robustness of SKI techniques, defined as a measure of performance degradation in response to systematic dataset variations. The proposed metric enables precise quantification of the robustness degree across injection approaches and different perturbations. Details on generating and quantifying perturbations are also provided. We evaluate the effectiveness of our metric through several experiments, where we apply multiple SKI techniques to three datasets and measure how robustness varies as perturbations increase.

keywords   Symbolic Knowledge Injection, Robustness, Neural Networks

origin event

 WOA 2024

journal or series

 CEUR Workshop Proceedings (CEUR-WS.org)

funding project

 FAIR-PE01-SP08 — Future AI Research – Partenariato Esteso sull'Intelligenza Artificiale – Spoke 8 “Pervasive AI” (01/01/2023–31/12/2025)

 ENGINES — ENGineering INtElligent Systems around intelligent agent technologies (28/09/2023–27/09/2025)

reference publication for talk

 An Empirical Study on the Robustness of Knowledge Injection Techniques Against Data Degradation (WOA 2024, 09/07/2024) — Andrea Agiollo (Andrea Rafanelli, Matteo Magnini, Andrea Agiollo, Giovanni Ciatto, Andrea Omicini)