You may never have heard of the term “slopsquatting,” but as AI continues to rapidly expand, it’s likely to be a term you’ll become very familiar with in the not-too-distant future.
AI capabilities continue to improve at a rapid pace, and one sector feeling the profound impact is software engineering. With a few prompts, AI can write code automatically, in a fraction of the time it would have taken a human coder. A recent survey found that 45% of organizations have launched AI-generated code. Some companies are even using AI to write as much as 75% of their code. Microsoft’s CTO predicted that 95% of code is going to be AI-generated within five years.
While there are clear productivity gains to be realized, using AI to generate code comes with potentially serious risks. “Slopsquatting” is one such security risk. In a slopsquatting attack, a malicious software package with a fictitious but plausible name is uploaded to online repositories. These packages contain pre-written code that developers commonly import into their own programs to accomplish certain tasks, rather than writing everything from scratch.
Here’s where it gets tricky. If an AI coding tool hallucinates the same package name as the malicious package instead of the authentic one, malware can be executed onto the users’ systems when the code is run. Like so much when it comes to malicious online activity, since the packages have seemingly legitimate names, it is difficult to detect, and the possibility exists to trick tens of thousands of unsuspecting software developers and vibe coders.
Academic researchers have found that between 5.2% and 22% of package names generated by AI are hallucinated, depending on the AI model.
As software code becomes more complex, with increased dependence on imported code libraries and packages, the attack surface grows, and software supply chains become more vulnerable. This is an example of a more general “third party risk.” Third-party risk refers to risk that an organization faces due to its relationships with its vendors, including software providers. An organization may have strong cybersecurity measures in place, but if an attacker finds and exploits a weakness of one of its vendors, the attacker may be able to use that entry point as a way to damage the target organization, resulting in negative impacts to cybersecurity, operations, finances, or reputation. Even though the risk wasn’t the target organization’s fault, they may still end up shouldering the consequences.
When it comes to risk management, four strategies are typically proposed: avoid, mitigate, transfer, and accept.
- Avoiding the risk of AI-generated code vulnerabilities would mean not using AI to generate code. Given the prevalence and growth of AI tools in the software engineering space, this may be an impractical risk management strategy.
- Mitigating a risk generally involves decreasing the probability of a risk’s occurrence, decreasing the consequences if it does occur, or both. Using automated tools for code quality analysis and code review can reduce the likelihood that malicious code is published. Self-healing, resilient systems that bounce back after disruption can help reduce system downtime, as well as fast incident response capabilities.
- Transferring risk may be achieved through contractual agreements that specify third-party responsibilities and expectations before and after a potential cyber incident. Cyber insurance, a rapidly evolving financial instrument, is another potential avenue, but policies may not cover every scenario.
- Accepting a risk does not mean inaction. Even when risks are retained, approaches like anomaly detection and continuous monitoring are used to scan systems for anything that could cause harm.
AI continues to make an impact across the economic landscape. In many cases, these impacts are mixed, and managers need to balance the pros and cons of AI implementation within their organization and across the supply chain. Guidance is beginning to emerge, such as the NIST AI Risk Management Framework, that provides a standardized structure and process for thinking about and managing the downsides associated with AI. Implementing strong risk management practices will help organizations adopting AI coding tools, and AI tools more broadly, to capture the upside benefits while minimizing the negative impacts of slop.
