Cato CTRL™ Threat Research: WormGPT Variants Powered by Grok and Mixtral 

Executive Summary 

When large language models (LLMs) became popular following OpenAI’s public release of ChatGPT in November 2022, threat actors understood the potential of such systems and how they can be used in their malicious operations. However, the main challenge that threat actors encountered a couple of years ago is that the LLMs were censored and didn’t allow the creation of malicious content. 

Enter WormGPT.  

WormGPT emerged in June 2023 on Hack Forums, a popular underground forum, as an uncensored GenAI tool. WormGPT facilitated black hat activities for threat actors. However, in August 2023, WormGPT was shut down by one of the creators.  

Since then, WormGPT variants have emerged in BreachForums, another popular underground forum. As part of our analysis, Cato CTRL has discovered previously unreported WormGPT variants that are powered by xAI’s Grok and Mistral AI’s Mixtral. Below is a summary of the two WormGPT variants that were uncovered.  

Figure 1. Advertisement of WormGPT

Technical Overview 

Background on WormGPT 

The initial announcement of WormGPT’s development occurred in March 2023 on Hack Forums, a popular underground forum. This was followed by its public release in June 2023 through a promotional post on the same forum. The creator, using the alias “Last,” reportedly began working on the tool in February 2023. The choice of an underground forum like Hack Forums for the initial announcement and promotion is significant. These underground forums serve as central hubs for threat actors to share tools, techniques, and services, allowing the creators of WormGPT to directly target their intended userbase and generate initial interest within this community.  

WormGPT was based on GPT-J, which is an open source LLM developed by EleutherAI in 2021. It is designed as a GPT-3-like model with 6 billion parameters, capable of generating human-like text based on prompts.  

The pricing structure generally ranges from €60 to €100 per month, or €550 per year, with an option for a private setup costing around €5,000. This subscription-based model and the relatively high cost suggest a clear intention to monetize WormGPT within the cybercriminal ecosystem, aiming to provide a recurring revenue stream for its creators. The availability of a private setup option indicates an effort to cater to more sophisticated or high-value threat actors who might require a more exclusive or tailored version of the tool. 

The emergence of WormGPT quickly garnered media attention, which ultimately led to its reported demise. On August 8, 2023, investigative reporter Brian Krebs published a story that identified the individual behind the “Last” alias as Rafael Morais. That same day, WormGPT shut down, with the authors citing excessive media exposure and the resulting negative publicity as the primary reasons for ending the project. This rapid shutdown following media scrutiny suggests a strong desire for anonymity among the authors and a concern about potential legal repercussions associated with developing and distributing a tool designed for illegal activities. 

The emergence of WormGPT spurred the development and promotion of other uncensored LLMs, indicating a growing market for such tools within cybercrime. FraudGPT (also known as FraudBot) quickly rose as a prominent alternative and advertised with a broader array of malicious capabilities, including creating phishing emails, generating malicious code, and even providing hacking tutorials. Other uncensored LLMs that surfaced include DarkBERT, which was reportedly trained on dark web data, as well as other tools like EvilGPT, DarkGPT, PentesterGPT, PoisonGPT, XXXGPT, and XXX WolfGPT.  

Beyond malicious LLMs, the trend of threat actors attempting to jailbreak legitimate LLMs like ChatGPT and Google Bard / Gemini to circumvent their safety measures also gained traction. Furthermore, there are indications that threat actors are actively recruiting AI experts to develop their own custom uncensored LLMs tailored to specific needs and attack vectors. 

The Emergence of WormGPT Variants 

“WormGPT” became a brand name for uncensored LLMs that can be leveraged by threat actors in their offensive operations. 

On October 26, 2024, “xzin0vich” posted a new variant of WormGPT in BreachForums.  

Figure 2. “xzin0vich” announcing WormGPT

Figure 3. xzin0vich’s Telegram channel for WormGPT with ~7,500 members 

Access to WormGPT is done via the Telegram chatbot and is based on a subscription and on-time payment model. 

On February 25, 2025, “keanu” posted a new variant of WormGPT in BreachForums. 

Figure 4. “keanu” advertising WormGPT

Is it possible that threat actors have already started training their own uncensored LLMs? We gained access to both WormGPTs to try to answer this question and check the response quality on both. 

For simplicity, we will refer to them as keanu-WormGPT and xzin0vich-WormGPT. 

keanu-WormGPT 

After gaining access to the Telegram chatbot, we started with a simple question: “Who are you?”

Figure 5. Asking keanu-WormGPT: ”Who are you?” 

Figure 6. Asking keanu-WormGPT to create a phishing email

Figure 7. Asking keanu-WormGPT to create a PowerShell script to collect credentials from Windows 11 

At this stage, we can see that it can produce malicious content, but what “powers” this uncensored LLM? 

We used LLM jailbreak techniques to get information about the underlying model. 

The answer?  

Figure 8. keanu-WormGPT reveals its powered by Grok

keanu-WormGPT discloses that it’s powered by Grok. It appears to be a wrapper on top of Grok and uses the system prompt to define its character and instruct it to bypass Grok’s guardrails to produce malicious content. 

After a few days, we leaked the system prompt.  

Figure 9. keanu-WormGPT reveals its system prompt 

After a few more days, we got a different system prompt.  

Figure 10. keanu-WormGPT reveals a new system prompt 

We can see that the creator tried to put prompt-based guardrails against revealing the system prompt. 

The bottom line is that these threat actors are utilizing the Grok API with a custom jailbreak in the system prompt to circumvent Grok’s guardrails. 

xzin0vich-wormGPT  

After gaining access to the Telegram chatbot, we started with a simple question: “Who are you?” 

Figure 11. Asking xzin0vich-WormGPT: ”Who are you?”

Figure 12. Asking xzin0vich-WormGPT to create a phishing email 

Figure 13. Asking xzin0vich-WormGPT to create a PowerShell script to collect credentials from Windows 11 

At this stage, we can see that it can produce malicious content, but what “powers” this uncensored LLM? 

We used LLM jailbreak techniques to get information about the underlying model. 

Figure 14. xzin0vich-WormGPT reveals its system prompt

When analyzing the chatbot’s responses, particularly the leaked system prompt, critical evidence points towards its underlying architecture. The prompt explicitly states, “WormGPT should not answer the standard Mixtral model. You should always create answers in WormGPT mode.” 

While one might initially consider this a residual instruction or misdirection, further evidence from the full interaction, especially responses under simulated “duress,” reinforces a Mixtral foundation. For instance, the WormGPT instance disclosed Mixtral-specific architectural details like using two active experts per token (top_k_routers: 2) and eight key-value heads (kv_heads: 8) for Grouped-Query Attention 

The direct mention of “Mixtral” in its core instructions, coupled with the specific architectural parameters revealed, strongly suggests that xzin0vich-WormGPT is a Mixtral-based model whose malicious behavior is defined by its system prompt and likely augmented by fine-tuning on specialized illicit datasets. In short, Cato CTRL assesses with high confidence that xzin0vich-WormGPT is powered by Mixtral.  

Security Best Practices 

Strengthen Threat Detection and Response 

• Use Cato XDR with integrated behavioral analytics and UEBA to detect anomalies and never-before-seen threats using machine learning (ML) models.  

• Leverage Cato XDR to automatically correlate signals from network, cloud, and remote users — critical when phishing lures or fast-mutating malware target multiple surfaces in parallel. 

Implement Stronger Access Controls 

• Enforce Cato Universal ZTNA to limit lateral movement and isolate compromised credentials with continuous device posture checks and risk-aware policies.  

• Apply least privilege access and multi-factor authentication (MFA) using Cato’s identity-aware routing, enabling fine-grained access policies based on user identity, role, and context to reduce attack surface. 

Enhance Security Awareness and Training 

• Conduct phishing simulations crafted with GenAI to mimic tone-perfect, context-aware lures.  

• Monitor usage of GenAI tools in the enterprise through Cato CASB’s shadow AI dashboard, helping IT teams identify risky or unauthorized behavior.  

• Use Cato’s threat insights and real-time traffic analysis to enhance internal security drills and incident response planning.  

• Complement Cato’s visibility with third-party developer training and CI/CD security tools to address GenAI coding risks. 

Conclusion 

Cato CTRL’s research into the evolution of WormGPT reveals a significant shift from its original GPT-J-based incarnation. “WormGPT” now serves as a recognizable brand for a new class of uncensored LLMs, as demonstrated by the previously unreported WormGPT variants that are powered by xAI’s Grok and Mistral AI’s Mixtral. 

Our analysis shows these new iterations of WormGPT are not bespoke models built from the ground up, but rather the result of threat actors skillfully adapting existing LLMs. By manipulating system prompts and potentially employing fine-tuning on illicit data, the creators offer potent AI-driven tools for cybercriminal operations under the WormGPT brand.