Conference Paper


Exploratory datamorphic testing of classification applications

Abstract

Testing has been widely recognised as difficult for AI applications. This paper proposes a set of testing strategies for testing machine learning applications in the framework of the datamorphism testing methodology. In these strategies, testing aims at exploring the data space of a classification or clustering application to discover the boundaries between classes that the machine learning application defines. This enables the tester to understand precisely the behaviour and function of the software under test. In the paper, three variants of exploratory strategies are presented with the algorithms as implemented in the automated datamorphic testing tool Morphy. The correctness of these algorithms are formally proved. The paper also reports the results of some controlled experiments with Morphy that study the factors that affect the test effectiveness of the strategies.

Attached files

Authors

Zhu Hong
Bayley, Ian

Oxford Brookes departments

School of Engineering, Computing and Mathematics

Dates

Year of publication: 2020
Date of RADAR deposit: 2020-05-28



© 2020 Association for Computing Machinery. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.


Related resources

This RADAR resource is Identical to Exploratory datamorphic testing of classification applications / Hong Zhu and Ian Bayley. AST '20: Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test. October 2020. Pages 51–60.

Details

  • Owner: Joseph Ripp
  • Collection: Outputs
  • Version: 1 (show all)
  • Status: Live