Marco Alderighi, Matteo Baldoni, Cristina Baroglio, Roberto Micalizio, Stefano Tedeschi (a cura di)
WOA 2024 – 25th Workshop "From Objects to Agents 2024", pp. 20–32
CEUR Workshop Proceedings (AIxIA Series) 3735
Sun SITE Central Europe, RWTH Aachen University
luglio 2024
Symbolic knowledge injection (SKI) represents a promising paradigm for bridging symbolic knowledge and sub-symbolic predictors in intelligent autonomous agents. Given the wide availability of SKI methods from the literature, we observe that SKI effectiveness is commonly measured in terms of predictive performance variation – e.g., accuracy improvement – introduced by SKI. However, other aspects such as the injection mechanism’s ability to maintain its performance and generalisation capability, despite encountering unexpected or anomalous input during the training process, are equally relevant. Accordingly, in this paper we propose a new metric to evaluate the robustness of SKI techniques, defined as a measure of performance degradation in response to systematic dataset variations. The proposed metric enables precise quantification of the robustness degree across injection approaches and different perturbations. Details on generating and quantifying perturbations are also provided. We evaluate the effectiveness of our metric through several experiments, where we apply multiple SKI techniques to three datasets and measure how robustness varies as perturbations increase.
parole chiave
Symbolic Knowledge Injection, Robustness, Neural Networks
evento origine
rivista o collana
progetto finanziatore
FAIR-PE01-SP08 — Future AI Research – Partenariato Esteso sull'Intelligenza Artificiale – Spoke 8 “Pervasive AI”
(01/01/2023–31/12/2025)
ENGINES — ENGineering INtElligent Systems around intelligent agent technologies
(28/09/2023–27/09/2025)
funge da
pubblicazione di riferimento per presentazione