How releasing the Llama 2 language model as open source could come back to bite Meta, and all of us, in the ass ... at least according to Llama 2
The AI bot is sounding a lot more sensible than Meta's AI bigwigs
On Tuesday, Meta launched the latest version of its Llama language model as open-source software, sending fans of DIY chatbots into a bit of a frenzy. As a (reluctant) AI doomer, the whole thing has me feeling a little shook. At a time when advanced AI development should be slowing (or getting slowed) the fuck down, Meta is hitting the accelerator—and putting a powerful language model in the hands of some of the worst people on planet Earth.
Meta, known for its mantra “move fast and break things,” is at least aware of the safety concerns inherent in open source; the paper that accompanied the release of the language model is full of discussions of possible dangers and ways to mitigate them. They’ve even provided a handy 24-page “Responsible Use Guide” based on the premise that “helping to ensure that generative AI technology does not produce undue harm is of paramount importance.” I’m pretty sure that none of the bad actors looking to exploit the open-source software are going to give this even a glance.
Nick Clegg—the former British politician turned Meta exec—offered a somewhat cruder response to critics of the open-source move. We don’t need to worry about existential risks just yet because these seemingly smart LLMs we have today are actually not so clever after all. In an interview with the BBC, Clegg declared that
the hype has somewhat run ahead of the technology. I think a lot of the existential warnings relate to models that don’t currently exist, so-called super-intelligent, super-powerful AI models – the vision where AI develops an autonomy and agency on its own, where it can think for itself and reproduce itself. The models that we’re open-sourcing are far, far, far short of that. In fact, in many ways they’re quite stupid.
This is more or less the company line at Meta, whose top AI guru, Yann LeCun, recently compared Meta’s software unfavorably to man’s best friend. “Worrying about superhuman AI alignment today is like worrying [about] turbojet engine safety in 1920,” he tweeted in April, adding that “we do not have a working design for anything that could come close to becoming as smart as a dog.”
I decided to take my questions about the open-sourcing of Llama 2 to someone who knows the language model perhaps better than anyone else. I’m speaking, of course, of Llama 2 itself, the 70 billion parameters version, which you can already access on Hugging Face’s HuggingChat site. And, for what it’s worth, it did a much better job explaining the risks than a dog could.
While Llama 2 wouldn’t say whether it thought releasing … itself open source was a good or a bad thing, overall, the chatbot was happy to spell out some of the risks we may face from the powerful language model falling into the hands of shitty people.
Spreading Disinformation: Bad actors could use LLaMA 2 to generate high-quality, convincing, and false information, such as fake news articles, propaganda, or deepfakes, which could be used to manipulate public opinion, sow confusion, and undermine trust in institutions.
Phishing Attacks: LLaMA 2 could be used to craft sophisticated phishing attacks, where bad actors could use the model to generate emails, texts, or social media posts that appear to come from legitimate sources, but actually aim to trick people into revealing sensitive information or installing malware.
Social Engineering: LLaMA 2 could be used to create persuasive and personalized messages to manipulate people's emotions, beliefs, and behaviors. This could be done through targeted advertising, propaganda, or even personalized attacks on individuals.
Malware Development: Bad actors could use LLaMA 2 to develop more sophisticated malware, such as viruses, Trojans, or ransomware, that can evade detection by traditional security systems.
Spear Phishing: LLaMA 2 could be used to create highly customized spear phishing attacks, where bad actors could use the model to generate emails, texts, or social media posts that appear to come from trusted sources, but actually aim to trick people into revealing sensitive information or installing malware.
Identity Theft: Bad actors could use LLaMA 2 to gather information about individuals, such as their interests, location, and relationships, which could be used to steal identities, commit fraud, or blackmail.
Financial Fraud: LLaMA 2 could be used to create sophisticated financial scams, such as Ponzi schemes, pyramid schemes, or phishing attacks, that could result in financial losses for individuals and organizations.
Cyberattacks: Bad actors could use LLaMA 2 to develop more sophisticated cyberattacks, such as SQL injection, cross-site scripting (XSS), or denial-of-service (DoS) attacks, that could compromise sensitive data and disrupt critical infrastructure.
Election Interference: LLaMA 2 could be used to influence elections by generating fake news, propaganda, or disinformation campaigns that aim to sway voter opinions or suppress turnout.
Psychological Manipulation: Bad actors could use LLaMA 2 to psychologically manipulate individuals, such as by using persuasive messaging to alter their beliefs, attitudes, or behaviors. This could be done through targeted advertising, propaganda, or even personalized attacks on individuals.
Now, it’s true that terrible humans are already using LLMs to help them do all these things, but making Llama 2 open-source could
make it easier for bad actors to use the model for malicious purposes, as they would have access to the model's architecture and parameters.
Specifically, Llama 2 noted, this could mean:
Easier accessibility: By making the model open source, anyone can download and use LLama 2 without needing to go through a vetting process or obtain permission from the developers. This could make it easier for bad actors to get their hands on the model and use it for nefarious purposes.
Customization: An open-source LLama 2 allows users to modify the model's architecture, parameters, and training data. While this can be useful for researchers and developers who want to improve the model, it also means that bad actors could potentially customize the model to suit their own needs, such as by adding bias or manipulating the output. …
It seems to be pretty easy to “customize.” One Redditor fiddled with Llama’s system prompt and some other parameters and managed to transform Llama 2 into a more, er, lively character called Amy:
And while Clegg and LeCun think that we won’t face existential risks from AI until some faraway time in the future, Llama 2 reminds us that it’s never too soon to start worrying about the possible deaths of every human being on Earth.
Yes, it is possible that releasing powerful large language models (LLMs) like LLama 2 as open source could increase the chances of extinction risks from AI. Here are some reasons why:
Uncontrolled proliferation: Once an LLM like LLama 2 is released as open source, it can be freely downloaded, copied, and modified by anyone. This could lead to an uncontrolled proliferation of the model, which could increase the risk of it falling into the wrong hands or being used for malicious purposes.
Improved capabilities: Open-source LLMs can be improved upon by anyone, which could lead to further advancements in their capabilities. While this could be beneficial in many ways, it could also increase the risk of the model being used for harmful purposes, such as creating convincing deep fakes or spreading disinformation.
Increased accessibility: By making LLMs like LLama 2 open source, they become more accessible to people who may not have had access to them otherwise. While this could democratize AI development and enable more people to benefit from these technologies, it could also increase the risk of the model being used by individuals or groups with malicious intentions.
Unintended consequences: As LLMs become more advanced and widespread, there is a risk of unintended consequences arising from their use. For example, an LLM designed to generate realistic text could be used to create convincing but false information, leading to confusion, misunderstandings, or even violence.
Lack of oversight: Releasing LLMs like LLama 2 as open source reduces the ability of governments, regulatory agencies, or ethical review boards to monitor and control their use. This lack of oversight could lead to the misuse of these powerful technologies, either intentionally or unintentionally.
Inadequate safety measures: Open-source LLMs may not include adequate safety measures to prevent their misuse. …
Increased risk of AI accidents: As LLMs become more complex and autonomous, there is a growing risk of AI accidents occurring due to unforeseen circumstances or edge cases. Open-source LLMs may be less likely to receive timely updates or bug fixes, which could exacerbate this issue.
Potential for weaponization: Advanced LLMs like LLama 2 could be weaponized by nation-states, terrorist organizations, or other malevolent entities to spread propaganda, undermine political opponents, or engage in cyber warfare. Open-source versions of these models may be more susceptible to such uses. …
Unclear long-term implications: It is difficult to predict the long-term implications of releasing powerful LLMs like LLama 2 as open source. As these models continue to evolve and improve, we may encounter novel risks and challenges that we cannot yet anticipate.
It seems more than a little shortsighted to dismiss existential threats as being so “long term” that we don’t need to worry about them now, much less let them affect what we do today. In the context of AI, long-term could mean 30 years; it could also mean three. In releasing Llama 2 as open source, Meta seems to be thinking only of the AI race that kicked into high gear after OpenAI unleashed ChatGPT on us last fall. I’m not sure history, however much of it we have left, will judge them kindly for this move.
Art by Midjourney