Machine learning and watermarking for accurate detection of AI generated phishing emails.

Lists

Brissett, Adrian and Wall, Julie ORCID: https://orcid.org/0000-0001-6714-4867 (2025) Machine learning and watermarking for accurate detection of AI generated phishing emails. Electronics, 14 (13). pp. 1-21.

Preview

PDF (PDF/A)
Machine Learning and Watermarking for Accurate Detection_WallJ_accessible.pdf - Published Version
Available under License Creative Commons Attribution.
Download (681kB) | Preview

Official URL: https://www.mdpi.com/2079-9292/14/13/2611

Abstract

Large Language Models offer transformative capabilities but also introduce growing cybersecurity
risks, particularly through their use in generating realistic phishing emails. Detecting such content is critical; however, existing methods can be resource-intensive and slow to adapt. In this research, we present a dual-layered detection framework that combines supervised learning for accurate classification with unsupervised techniques to uncover emerging threats. In controlled testing environments, our approach demonstrates strong performance. Recognising that human users are often the weakest link in information security systems, we examine historical deception patterns and psychological principles commonly exploited in phishing attacks. We also explore watermarking as a complementary method for tracing AI-generated content. Together, these strategies offer a scalable, adaptive defence against increasingly sophisticated phishing attacks driven by Large Language Models.

Item Type:	Article
Identifier:	10.3390/electronics14132611
Keywords:	phishing detection; large language models; AI-generated content; watermarking; techniques; paraphrasing detection; hybrid detection models
Subjects:	Computing
Depositing User:	Julie Wall
Date Deposited:	04 Jul 2025 14:02
Last Modified:	04 Jul 2025 14:15
URI:	https://repository.uwl.ac.uk/id/eprint/13810

Downloads

Downloads per month over past year

Actions (login required)

View Item

References

Stokel-Walker, C. (Ed.) The AI Revolution: What the New Age of Artificial Intelligence Means for Humanity; New Scientist Essential Guide No. 23; New Scientist: London, UK, 2024.
Kucharavy, A. From Deep Neural Language Models to LLMs. In Large Language Models in Cybersecurity: Threats, Exposure and Mitigation; Springer Nature: Cham, Switzerland, 2024; pp. 3–17.
Erbschloe, M. Social Engineering: Hacking Systems, Nations, and Societies, 1st ed.; CRC Press: Boca Raton, FL, USA, 2019. https://doi.org/10.1201/9780429322143.
Jakobsson, M.; Myers, S. Phishing and Countermeasures, 1st ed.;Wiley-Interscience: Oxford, UK, 2006.
Nunes, V. The Cyber Skill Gap: How to Become a Highly Paid and Sought After Information Security Specialist; Self-Published, 2017; Available online: https://www.amazon.com/Cyber-Skill-Gap-Information-Specialist-ebook/dp/B06XJD1W7N(accessed on 10 April 2024).
Gallagher, S.K.; Ratchford, J.; Brooks, T.; Brown, B.P.; Heim, E.; Nichols,W.R.; Mcmillan, S.; Rallapalli, S.; Smith, C.J.; VanHoudnos, N.; et al. Assessing LLMs for High Stakes Applications. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, Lisbon Portugal, 14–20 April 2024; pp. 103–105.
Elkhatat, A.M.; Elsaid, K.; Almeer, S. Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. Int. J. Educ. Integr. 2023, 19, 17..
Greco, F.; Desolda, G.; Esposito, A.; Carelli, A. David versus Goliath: Can Machine Learning Detect LLM-Generated Text? A Case Study in the Detection of Phishing Emails. In Proceedings of the ITASEC 2024: The Italian Conference on CyberSecurity, Salerno, Italy, 9–11 April 2024.
Sadasivan, V.S.; Kumar, A.; Balasubramanian, S.;Wang,W.; Feizi, S. Can AI-Generated Text be Reliably Detected? arXiv 2024, arXiv:2303.11156.
Ghosal, S.S.; Chakraborty, S.; Geiping, J.; Huang, F.; Manocha, D.; Bedi, A.S. Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey. arXiv 2023, arXiv:2310.15264.
Tao, Z.; Li, Z.; Xi, D.; Xu, W. CUDRT: Benchmarking the Detection of Human vs. Large Language Models Generated Texts. arXiv 2024, arXiv:2406.09056v1.
Cialdini, R.B. Influence: The Psychology of Persuasion, rev., ed.; Harper Business: New York, NY, USA, 2007.
Gutmann, A. An Analysis of Computer Systems for the Secure Creation and Verification of User Instructions. Ph.D. Thesis, University College London, London, UK, 2020.
Hazell, J. Spear Phishing with Large Language Models. arXiv 2023, arXiv:2305.06972.
Chapple,M.; Seidl, D. Cyberwarfare: Information Operations in a ConnectedWorld; Jones & Bartlett Learning: Burlington,MA, USA, 2021.
Firdhous, M.F.M.; Elbreiki,W.; Abdullahi, I.; Sudantha, B.H.; Budiarto, R. Wormgpt: a large language model chatbot for criminals. In Proceedings of the 24th International Arab Conference on Information Technology (ACIT), Ajman, United Arab Emirates, 6–8 December 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1–26.
Da Silva Gameiro,H. LLMDetectors. In Large LanguageModels in Cybersecurity: Threats, Exposure andMitigation; SpringerNature: Cham, Switzerland, 2024; pp. 197–204.
Gehrmann, S.; Strobelt, H.; Rush, A.M. GLTR: Statistical Detection and Visualization of Generated Text. arXiv 2019, arXiv:1906.04043.
Guo, B.; Zhang, X.; Wang, Z.; Jiang, M.; Nie, J.; Ding, Y.; Yue, J.; Wu, Y. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. arXiv 2023, arXiv:2301.07597.
Bateman, J. Deepfakes and Synthetic Media in the Financial System: Assessing Threat Scenarios; Carnegie Endowment for International Peace: Washington, DC, USA, 2020.
Weber-Wulff, D.; Anohina-Naumeca, A.; Bjelobaba, S.; Folt `ynek, T.; Guerrero-Dib, J.; Popoola, O.; Šigut, P.; Waddington, L. Testing of detection tools for AI-generated text. Int. J. Educ. Integr. 2023, 19, 26 .
Bhattacharjee, A.; Liu, H. Fighting Fire with Fire: Can ChatGPT Detect AI-Generated Text? ACM SIGKDD Explorations Newsletter 2024, 25, 14–21, ACM New York, USA.
Kirchner, D.; Reiter, A.; Scholkopf, B. Watermarking LLM-Generated Texts via Synonym Substitution: Preserving Semantics while Enabling Detection. arXiv 2023, arXiv:2306.04634.
Manning, C.D.; Raghavan, P.; Schütze, H. Introduction to Information Retrieval; Cambridge University Press: Cambridge, UK, 2008.
Demir, M.; Yildirim, M. Efficient Hybrid Movie Recommendation System Framework Based on A Sequential Model. 2023.
Available online: https://www.researchgate.net/publication/372490095 (accessed on 21 April 2025).
Jurafsky, D.; Martin, J.H. Speech and Language Processing, 3rd ed.; Pearson: London, UL, 2021; p. 199.
Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient Estimation of Word Representations in Vector Space. arXiv 2013, arXiv:1301.3781.
Trappenberg, T.P. Fundamentals of machine learning; Oxford University Press: Oxford, UK, 2019.
ISO/IEC 42001:2023; Information Technology—Artificial Intelligence—Management System. ISO: Geneva, Switzerland, 2023.

Tools

CORE (COnnecting REpositories)

The University of West London

Machine learning and watermarking for accurate detection of AI generated phishing emails.

Abstract

Downloads

Actions (login required)

Menu