Abstract: Artificial neural network (ANN) ensembles has long suffered from lack of interpretability. This has severely limited the practical usability of ANNs in settings where an erroneous decision can be disastrous. Several attempts have been made to alleviate this problem. Many of them are based on decomposing the decision boundary of the ANN into a set of rules.

We explore and compare a set of new methods for this explanation process on two artificial data sets (Monks 1 and 3), and one acute coronary syndrome data set consisting of 861 electrocardiograms (ECG) collected retrospectively at the emergency department at Lund University Hospital.

The algorithms managed to extract good explanations in more than 84% of the cases. More to the point, the best method provided 99 and 91% good explanations in Monks data 1 and 3 respectively. Also there was a significant overlap between the algorithms. Furthermore, when explaining a given ECG, the overlap between this method and one of the physicians was the same as the one between the two physicians in this study. Still the physicians were significantly, p-value < 0.001, more similar to each other than to any of the methods.

The algorithms has the potential to be used as an explanatory aid when using ANN ensembles in clinical decision support systems.