An Empirical Study on the Robustness of Knowledge Injection Techniques Against Data Degradation

Symbolic knowledge injection (SKI) represents a promising paradigm for bridging symbolic knowledge and sub-symbolic predictors in intelligent autonomous agents. Given the wide availability of SKI methods from the literature, we observe that SKI effectiveness is commonly measured in terms of predictive performance variation – e.g., accuracy improvement – introduced by SKI. However, other aspects such as the injection mechanism’s ability to maintain its performance and generalisation capability, despite encountering unexpected or anomalous input during the training process, are equally relevant. Accordingly, in this paper we propose a new metric to evaluate the robustness of SKI techniques, defined as a measure of performance degradation in response to systematic dataset variations. The proposed metric enables precise quantification of the robustness degree across injection approaches and different perturbations. Details on generating and quantifying perturbations are also provided. We evaluate the effectiveness of our metric through several experiments, where we apply multiple SKI techniques to three datasets and measure how robustness varies as perturbations increase.