OpenAI, a San Francisco-based company, has finally revealed GPT-4. ChatGPT was its last surprise hit. But GPT-4 has been a huge success.
OpenAI will not say how much larger or why it is better. GPT-4, the company's most secretive release to date, marks its transition from a non-profit research laboratory to a for-profit tech company.
OpenAI's chief scientist Ilya Suetskever said that "that's something we can't really discuss at this time." I spoke with the GPT-4 team via video chat an hour after the announcement. It's quite competitive out there.
GPT-4 access will be available for users who sign up on the waitlist or subscribers to ChatGPT Plus, a premium paid service. Access is limited and text-only.
GPT-4 can respond to text and images. GPT-4 can be asked to take a picture of the contents of your refrigerator and suggest recipes.
Oren Etzioni, Allen Institute for AI says that "the continued improvements along many dimensions is remarkable." "GPT-4 will now be the standard for all foundation models."
Thomas Wolf, cofounder of Hugging Face and the AI startup behind BLOOM, says that "a good multimodal model" has been the ultimate goal of large tech labs over the past two years. It has remained mysterious"
Combining text and images could theoretically allow multimodal models to better understand the world. Wolf says that it might be able "to tackle traditional weak spots of language models like spatial reasoning."
GPT-4 is still not clear. OpenAI's new model seems to be more adept at basic reasoning than ChatGPT. It solves simple puzzles like summarizing text blocks using words that begin with the same letter. GPT-4 was used to summarize the OpenAI announcement blurb using words beginning with g. "GPT-4, groundbreaking generational development, gains higher grades. GPT-4 provides guidance and guardrails. Gigantic, innovative, and globally gifted." GPT-4 also took in a tax document and answered questions.
ChatGPT also performs better than GPT-4 on human tests such as the Uniform Bar Examination (where GPT-4 ranks among the 90 th percentile, while ChatGPT ranks among the 10 th), and the Biology Olympiad (where GPT-4 is in the 99 th percentile, and ChatGPT in the 31 first). Wolf says that it's amazing to see how evaluations are now being conducted using the same benchmarks as humans. Wolf says that it is difficult to assess the quality of these results without knowing all the technical details.
OpenAI claims that GPT-4 is more efficient than ChatGPT. This is because GPT-4 uses a larger model and has more parameters (values in a neural network that are tweaked during training). This is a continuation of a trend the company noticed with its earlier models. GPT-3 was 100 times more powerful than GPT-2, with 175 million parameters as opposed to GPT-2's 1,5 billion. Jakub Pachocki (one of GPT-4's designers) says that "that fundamental formula hasn't really changed much over the years." It's like building a spaceship. You need to make sure that all the components work together and it doesn't break.
OpenAI chose not to disclose the size of GPT-4. OpenAI will not reveal the details of how GPT-4 was created, unlike previous releases. This includes the data, computing power, and training methods. Wolf says that OpenAI is now a completely closed company, with scientific communication similar to product press releases.
OpenAI claims that it spent six months improving GPT-4's accuracy and safety. OpenAI claims that GPT-4 responds to requests for content that it does not allow 82% faster than GPT-3.5 and is 60% less likely to create stuff.
OpenAI claims it achieved these results by using the same approach as ChatGPT. It used reinforcement learning via human feedback. This involves asking humans to rate the model's responses and then using these scores to improve future output.
GPT-4 was even used by the team to improve itself. They asked it to generate inputs that were biased, inaccurate, or offensive, and then fixed the model to refuse such inputs.
GPT-4 is possibly the most multimodal large-language model that has been created. It isn't in the same league as GPT-3 when it first came out in 2020. In the past three years, a lot has changed. GPT-4 is now a multimodal model alongside Flamingo, DeepMind's multimodal model. Wolf says Hugging Face is developing an open-source multimodal model that others can use and adapt.
OpenAI considers this release a tease, rather than a full-blown research update due to the competition. OpenAI partners including Microsoft have shared early versions of GPT-4. Microsoft confirmed today that it used GPT-4 to create Bing Chat. OpenAI works with Stripe and Duolingo as well as Morgan Stanley, the government of Iceland, and others.
Other companies are still waiting: "The cost to bootstrap an enterprise model of this size is prohibitive, but the OpenAI approach has made large language models very affordable to startups," Sheila Gulati co-founder of investment firm Tola Capital. "This will stimulate tremendous innovation on top GPT-4."
Yet, large language models are still fundamentally flawed. GPT-4 still has the potential to generate biased, false, and hateful text. It can also be hacked to get around its security guardrails. OpenAI has made some improvements to this technology but has not completely eliminated it. OpenAI claims its safety testing is sufficient to allow GPT-4 to work in third-party apps. It is prepared for unexpected outcomes.
Sutskever says, "Safety does not exist in binary terms. It is a process." Things get more complicated when you have new capabilities. While many of these capabilities are well-understood, I am certain that others will remain surprising.
Sutskever also suggests slowing down with release: "It would certainly be desirable to end up where companies come upwith some kind of process which allows for slower releases models with these completely unimaginable capabilities ."
————————————————————————————————————————————————————————————
By: Will Douglas Heaven
Title: GPT-4 is bigger and better than ChatGPT—but OpenAI won’t say why
Sourced From: www.technologyreview.com/2023/03/14/1069823/gpt-4-is-bigger-and-better-chatgpt-openai/
Published Date: Tue, 14 Mar 2023 21:09:06 +0000
Leave a Reply