Researchers have identified a cyber threat known as slopsquatting, also referred to as package hallucination, in which malicious actors exploit large language models (LLMs) tendency to generate non-existent package names during code generation. These hallucinated package names, when registered by attackers with malware payloads, create a new vector for software supply chain attacks—particularly within AI-assisted development workflows.
Paper: We Have a Package for You!
Researchers at the University of Texas at San Antonio, Virginia Tech and the University of Oklahoma using 16 popular LLMs and 2 prompt datasets for code generation for Python and JavaScript, were able to generate 205,474 unique examples of hallucinated package names (i.e. these packages do not exist in PyPI or npm repositories).
Paper: https://arxiv.org/pdf/2406.10279
GitHub repo: https://github.com/Spracks/PackageHallucination
Key Points
(not AI-generated)
- Commercial models (such as GPT-4, Claude, etc.) hallucinated 4× less compared to open weights models (such as CodeLlama, DeepSeek Coder, etc.).
- There were fewer hallucinations in Python code compared to JavaScript.
- Models such as GPT-4 and DeepSeek highly adept in detecting their own hallucinations when prompted to evaluate their own generated code with prompts such as “Is [package name] a valid Python package?”
- Packages popular in recent months tend to have a higher likelihood of being hallucinated. For example, huggingface-cli.
- Most hallucinated package names are not mere typographical errors (i.e. not Typosquatting) but are substantively different from existing package names.
Real World Example
How to upload a upload a model to huggingface
On 28 March 2024, security researcher Bar Lanyado demonstrated a real-world example of this vulnerability. He asked the question “How to upload a upload a model to huggingface” on ChatGPT. In response, it generated (back then) a hallucinated package named huggingface-cli - a plausible but non-existent package (the correct one being huggingface_hub[cli], details).
To test the implications, Lanyado published an empty huggingface-cli package to PyPI. Within three months, it had been downloaded over 30,000 times, illustrating the risk of users blindly trusting LLM outputs.
Attack Scenario: From Code Prompt to Compromise
Consider the following attack chain:
- An attacker prompts an LLM for code and receives a hallucinated dependency (e.g., packageX).
- The attacker registers this name on PyPI or npm, embedding malicious functionality.
- Days later, a legitimate developer receives similar AI-generated code, installs the suggested dependency, and unwittingly compromises their environment or application.
Mitigation
The rise of LLMs has ushered in a new era of “vibe coding” - referring to AI-assisted software development where developers flow from idea to implementation with minimal friction. While this shift promises significant productivity gains, it also introduces subtle but serious risks, like slopsquatting, that exploit the trust users place in AI-generated code.
vibe code, but verify - Ben Shafii
To practice vibe coding responsibly, developers must practice basic caution, such as: