Are LLMs Transforming Cybercriminal Operations? A Study of LLMs on Dark Web Marketplaces
This project examines the emergence of “dark LLMs,” AI models marketed or shared on the dark web for activities such as phishing, malware development, fraud, and other forms of cybercrime. It maps this growing ecosystem, develops a typology of these tools and their intended uses, and assesses whether they are meaningfully transforming the accessibility, scale, and sophistication of cybercriminal operations.
This project was generously funded by the UK AI Security Institute from 2025-2026.
About the Project
Recent innovations in artificial intelligence (AI) and Large Language Models (LLMs) have transformed many aspects of society—including crime. Criminals have creatively adapted widely available AI applications to develop malware, create customized phishing attacks, analyze intelligence gathered from surveillance, and even draft text for romance scams. However, despite several notable incidents, some skeptics contend that the use of LLMs in facilitating cybercrime is overstated, driven more by speculation rather than by measurable evidence.
One way to assess the use of AI and LLMs in cybercrime is by examining the illegal marketplaces where these tools are bought and sold. In recent years, cybercriminal activity—carried out by state-backed actors, organized criminal groups, and individuals—has become increasingly professionalized and democratized. With the rise of ransomware-as-a-service and dark web markets that sell exploits, malware, and other tools, cybercrime has become more accessible, scalable, and sophisticated. Accordingly, the emergence of LLMs within this context aligns with these broader trends, democratizing access to these tools and increasing the reach and complexity of cybercriminal operations.
This trend is already beginning to take shape. Several so-called “dark LLMs” have emerged for sale on dark web markets, with prices ranging from $50-$5,000. Models such as FraudGPT, WormGPT, DarkGPT, and ruGPT are marketed specifically for malicious use, including creating targeted phishing messages, developing malware, and generating child sexual abuse material. Despite the growing marketplace for dark LLMs, more systematic research is necessary to better quantify, measure, and understand the emergence and impact of these tools on cybercriminal operations.
This project aims to build a centralized body of knowledge that tracks the rise of dark LLMs for sale or shared on the dark web and intended for cybercriminal use. Specifically, we aim to: (1) map the presence of dark LLMs; (2) create a typology of dark LLMs, including their intended uses; (3) assess the utility and application of dark LLMs within cybercriminal operations; and (4) offer an assessment of dark LLMs’ potential to transform the cybercriminal ecosystem.
Workshop
In April 2026, CSINT and the Middlebury Institute of International Studies at Monterey convened a two-day workshop at EADA Business School in Barcelona, bringing together scholars working at the intersection of artificial intelligence and cybercrime. While the diffusion of AI into criminal ecosystems is widely assumed, the actual adoption patterns — which tools are used, by whom, and to what effect — remain undertheorized and thinly evidenced. Papers presented across three sessions examined AI-enabled state revenue operations and influence campaigns, the shifting organizational and economic dynamics of cybercriminal networks, the fragmented international regulatory picture, as well as our research on the jailbreak techniques circulating in underground forums.
Read the full agenda here.
Publications and Conferences
Stay tuned for the articles that come out of this research. In the meantime, you can catch our paper presentation at the Stanford Trust and Safety Research Conference in Fall 2027.
Uncharted Waters: Governing AI Infrastructure in a Water Scarce World
It All Begins Here
Confidence doesn’t always arrive with a bold entrance. Sometimes, it builds quietly, step by step, as we show up for ourselves day after day. It grows when we choose to try, even when we’re unsure of the outcome. Every time you take action despite self-doubt, you reinforce the belief that you’re capable. Confidence isn’t about having all the answers — it’s about trusting that you can figure it out along the way.
The key to making things happen isn’t waiting for the perfect moment; it’s starting with what you have, where you are. Big goals can feel overwhelming when viewed all at once, but momentum builds through small, consistent action. Whether you’re working toward a personal milestone or a professional dream, progress comes from showing up — not perfectly, but persistently. Action creates clarity, and over time, those steps forward add up to something real.
You don’t need to be fearless to reach your goals, you just need to be willing. Willing to try, willing to learn, and willing to believe that you’re capable of more than you know. The road may not always be smooth, but growth rarely is. What matters most is that you keep going, keep learning, and keep believing in the version of yourself you’re becoming.
Mainstreaming Trust & Safety in Online Gaming
In collaboration with the Center for Democracy & Technology and the NYU Stern Center for Business and Human Rights, this project examined how trust and safety practices in online games can inform the governance of digital platforms more broadly. The collaboration brought together experts from industry, civil society, government, and academia for a public workshop and produced a series of five essays and a white paper on pro-social design, content moderation, privacy, child safety, and community governance.
This work was generously supported by the CTD and the NYU Stern Center for Business and Human Rights
Online games are important spaces for entertainment, creativity, and social connection, yet they remain largely absent from mainstream debates about digital rights, platform governance, and online safety. In collaboration with the Center for Democracy & Technology and the NYU Stern Center for Business and Human Rights, this project explored how the experiences of the gaming industry could inform more effective and rights-respecting approaches to trust and safety across digital platforms.
The collaboration began with a public workshop that brought together experts from the gaming industry, civil society, government, and academia to examine the distinctive trust and safety challenges facing online games. Discussions focused on how games and social media confront many of the same problems, including harassment, privacy risks, harmful design practices, child safety, and content that moves across platforms, while also considering what each sector could learn from the other.
The project produced a series of five expert essays examining pro-social and universal design, privacy-preserving moderation, youth well-being, and community-led governance. It also resulted in the white paper Trust, Play, and Platforms: Sharing Lessons for Safer Digital Spaces, co-authored with Dean Jackson. Together, these publications argue for moving beyond reactive and punitive approaches to content moderation toward systems that build safety into platform design, support healthy community norms, protect user autonomy, and recognize the social benefits of online play.
Event poster, Mainstreaming Trust & Safety in Online Games