Maltby, Harry, Wall, Julie ORCID: https://orcid.org/0000-0001-6714-4867, Glackin, Cornelius, Moniri, Mansour, Salami, Iwa, Li, Letian and Cannings, Nigel (2026) Robustness in deepfake speech detection: A survey of failure mechanisms including an experimental case study. Computer Speech & Language, 102. pp. 1-24. ISSN 0885-2308

PDF/A WallJ_Robustness in deepfake speech detection_VoR_PDFA.pdf - Published Version Restricted to Repository staff only Download (2MB)

Abstract

In recent years, the emergence of disruptive deepfake technology (referring here to computer-generated speech/audio, images, video and text) has raised significant concerns surrounding the privacy, security, and credibility of digital content. Although detection systems report high benchmark performance, most are evaluated on in-domain datasets, codecs, and generative models seen during training. In real deployment, however, detectors must contend with out-of-domain instances, that is, a mismatch between the data seen during training and the conditions encountered in deployment (e.g., unseen generative methods, unfamiliar codecs, or signals degraded by transmission channels). Under these conditions, performance often degrades sharply.
While robustness has been acknowledged in the literature, it remains underexplored. This work presents a robustness-focused survey and experimental study of deepfake speech detection. Rather than broadly cataloguing existing methods, we organise the literature around three factors that influence robustness to distribution shift: dataset design, feature representations, and model architecture. To complement this synthesis, we conduct a cross-dataset evaluation with Wav2Vec2, HuBERT, WavLM, and Whisper self-supervised speech models, trained on the ASVspoof 2019 LA and evaluated on the ASVspoof 2021 Deepfake dataset.
The experimental results show that robustness varies considerably across models and conditions. HuBERT demonstrates comparatively stable performance across codec and dataset shifts, while Wav2Vec2 and WavLM experience substantial degradation, particularly under MP3-based compression and neural vocoder synthesis. A diagnostic analysis further reveals that robustness failures are strongly associated with codec-induced masking of synthesis artefacts and with dataset-dependent synthesis pipelines.
By combining a robustness-oriented survey with cross-dataset experimental evidence, this work puts theory to practice, highlighting key factors that limit the generalisation of current detectors and outlining practical directions for improving robustness, including diversified dataset design, feature fusion, and evaluation under realistic cross-domain conditions.

Item Type:	Article
Identifier:	10.1016/j.csl.2026.102037
Keywords:	Deepfake speech; Spoofed speech; Speech deepfake detection; Deep learning; Synthetic speech detection; Automatic speaker verification; Speech anti-spoofing; Machine learning
Subjects:	Computing > Information security Computing > Intelligent systems
Date Deposited:	05 Aug 2026
Dates:	Date Publication status 18 October 2025 Submitted 22 July 2026 Accepted 28 July 2026 Published Online
School, department or research centre:	CAINT (The Centre for AI and Natural Language Technologies) School of Computing and Engineering
Keywords:	Deepfake speech; Spoofed speech; Speech deepfake detection; Deep learning; Synthetic speech detection; Automatic speaker verification; Speech anti-spoofing; Machine learning
URI:	https://repository.uwl.ac.uk/id/eprint/15273
Sustainable Development Goals:	Goal 9: Industry, Innovation, and Infrastructure

Actions (admin access)