Meta has a big bet on open-source AI. The company has today released LLaMA 2 – its first large-scale language model, which is available to anyone for free.
Tech companies are racing to release new models since OpenAI's AI chatbot ChatGPT was released in November last year. They hope to overthrow its dominance. Meta is in the back seat. Meta released a smaller version of LLaMA in February, when Microsoft and Google announced AI chatbots. This was restricted to researchers. Meta hopes to catch up by releasing LLaMA 2 and allowing anyone to create commercial products based on it.
The company has released a set of AI models that includes versions of LLaMA 2 of different sizes as well as a version where people can create a chatbot similar to ChatGPT. The model is not available through OpenAI, but rather from Meta's launch partner Microsoft Azure, Amazon Web Services and Hugging Face.
Ahmad Al-Dahle ,, a Meta vice president who leads the company's work on generative AI says: "This is a big moment for us." This is a big, huge moment for us.
There are still many caveats. Meta does not release information about the data it used to train LLaMA 2; it cannot guarantee it did not include copyrighted or personal data. This is according to a research paper that was shared exclusively with MIT Technology Review. LLaMA 2 has the same issues as all large language models, including a tendency to create falsehoods and offensive words.
Al-Dahle explains that Meta hopes to learn valuable lessons by releasing its model and letting companies and developers play with it. This will help Meta make their models more accurate, efficient, and less biased.
Percy Liang is the director of Stanford’s Center for Research on Foundation Models. He believes that a powerful open-source language model such as LLaMA 2 could pose a significant threat to OpenAI. Liang was a member of the research team that developed Alpaca as an open-source alternative to GPT-3 – an earlier version OpenAI’s language model.
Liang says that "LLaMA2 isn't GPT-4." In its research paper Meta acknowledges that there is a significant performance gap between LLaMA 2 compared to GPT-4 which is OpenAI's current state-of-the art AI language model. He adds that "for many use cases you don't even need GPT-4".
He says that a more transparent and customizable model such as LLaMA 2 could help companies develop products and services quicker than a large, complex proprietary model.
Steve Weber, a University of California Berkeley professor, says that having LLaMA 2 as the leading alternative open-source to OpenAI is a big win for Meta.
Under The Hood
Al-Dahle states that LLaMA 2 required a great deal of work to get it ready for launch. This was to make it safer and less likely than its predecessor to spread toxic falsehoods.
Meta can learn from its past mistakes. After only three days its language model for science was taken down, Galactica. Its previous LlaMA, meant for research, was also leaked. This prompted politicians to question whether Meta had properly considered the risks of AI language models such as misinformation and harassment.
Meta used a combination of machine learning techniques to improve safety and helpfulness.
Sasha Luccioni is a researcher with Hugging Face and says that Meta's training LLaMA 2 model had more steps in it than normal for generative AI.
The model was trained with 40% more data than the predecessor. Al-Dahle said there were two types of data used for training: online data scraped, and data fine-tuned by human annotators based on feedback. The company claims it didn't use Meta user data for LLaMA 2 and excluded data from websites it knew contained a lot of personal information.
LLaMA 2 is no different from rival models in that it uses offensive, harmful and other problematic language. Meta claims it didn't remove the toxic data because it might help LLaMA 2 better detect hate speech, and removing them could result in accidentally filtering some demographic groups.
Luccioni says that Meta's openness is still exciting because it allows researchers to properly study AI models' biases and ethics.
Al-Dahle says that the fact that LLaMA 2 has an open-source design will allow researchers and developers from outside to test it for security vulnerabilities, making it safer than proprietary models.
Liang agrees. He says, "I am very excited to test things out and think that it will benefit the community."
————————————————————————————————————————————————————————————
By: Melissa Heikkilä
Title: Meta’s latest AI model is free for all
Sourced From: www.technologyreview.com/2023/07/18/1076479/metas-latest-ai-model-is-free-for-all/
Published Date: Tue, 18 Jul 2023 16:00:00 +0000
Leave a Reply