Symbolic knowledge extraction from opaque ML predictors in PSyKE: Platform design & experiments


Federico Sabbatini, Giovanni Ciatto, Roberta Calegari, Andrea Omicini

Intelligenza Artificiale 16(1), pages 27–48, 22 pages, July 2022
IOS Press
Roberta Calegari, Giovanni Ciatto, Andrea Omicini, Giuseppe Vizzari (eds.)

A common practice in modern explainable AI is to post-hoc explain black-box machine learning (ML) predictors – such as neural networks – by extracting symbolic knowledge out of them, in the form of either rule lists or decision trees. By acting as a surrogate model, the extracted knowledge aims at revealing the inner working of the black box, thus enabling its inspection, representation, and explanation. Various knowledge-extraction algorithms have been presented in the literature so far. Unfortunately, running implementations of most of them are currently either proofs of concept or unavailable. In any case, a unified, coherent software framework supporting them all – as well as their interchange, comparison, and exploitation in arbitrary ML workflows – is currently missing. Accordingly, in this paper we discuss the design of PSyKE, a platform providing general-purpose support to symbolic knowledge extraction from different sorts of black-box predictors via many extraction algorithms. Notably, PSyKE targets symbolic knowledge in logic form, allowing the extraction of first-order logic clauses. The extracted knowledge is thus both machine- and human-interpretable, and can be used as a starting point for further symbolic processing—e.g. automated reasoning.

(keywords) Explainable AI, knowledge extraction, interpretable prediction, PSyKE
 @article{psyke-ia16,
author = {Sabbatini, Federico and Ciatto, Giovanni and Calegari, Roberta and Omicini, Andrea},
doi = {10.3233/IA-210120},
editor = {Calegari, Roberta and Ciatto, Giovanni and Omicini, Andrea and Vizzari, Giuseppe},
iris = {11585/890822},
issn = {1724-8035},
issn-online = {2211-0097},
journal = {Intelligenza Artificiale},
month = jul,
number = 1,
pages = {27--48},
publisher = {IOS Press},
title = {Symbolic knowledge extraction from opaque {ML} predictors in {PSyKE}: Platform design \& experiments},
url = {https://content.iospress.com/articles/intelligenza-artificiale/ia220141},
volume = 16,
year = 2022

Journals & Series

Events

  • 22nd Workshop “From Objects to Agents” (WOA 2021) — 01/09/2021–03/09/2021

Publication

— authors

Federico Sabbatini, Giovanni Ciatto, Roberta Calegari, Andrea Omicini

— editors

Roberta Calegari, Giovanni Ciatto, Andrea Omicini, Giuseppe Vizzari

— status

published

— sort

article in journal

Venue

— journal

Intelligenza Artificiale

— volume

16

— issue

1

— pages

27–48

— publication date

July 2022

URLs

original page

Identifiers

— DOI

10.3233/IA-210120

— IRIS

11585/890822

— print ISSN

1724-8035

— online ISSN

2211-0097

BibTeX

— BibTeX ID
psyke-ia16
— BibTeX category
article

Partita IVA: 01131710376 - Copyright © 2008-2022 APICe@DISI Research Group - PRIVACY