OpenAIThere were plans to develop a new AI model called "Arrakis" that aims to make AI inference more efficient and affordable. However, due to quality issues, the company had to stop the development of this project.
Arrakis was supposed to be as powerful as GPT-4, but cheaper, because the model partially exploits what's known as the sparsity principle. This means that only parts of the neural network are used to process the input, whereas in traditional "dense models," the entire neural network is active. Google, for example, has used the sparsity principle in its Pathways AI project.
Source Note: The image is generated by AI, and the image is authorized by Midjourney
The Information cited two people familiar with the project as saying that the development of the "Arrakis" project began last fall and training began in the spring. But OpenAI's development team soon realized that the performance was not good enough.
The team made some fine-tuning over a month, and then leadership killed the project, but it’s unclear why the sparsity principle worked well in early tests but failed on larger models.
Despite this setback, OpenAI may apply the lessons learned from Project Arrakis to other models, such as the planned Gobi multimodal model.After the failure of Project Arrakis, OpenAI researchers set out to develop a version of GPT-4 that could generate faster responses.
This is likely why GPT-4 version “0613”, which was released this summer, was significantly faster than version “0314” that OpenAI released in early March. Although some users complained about some loss of quality in certain areas, OpenAI refuted this, which led to the company taking longer than originally planned to provide API access to version 0314.
OpenAI is taking artificial intelligence to the desert, naming their new AI model prototypes "Gobi", "Sahara" and "Arrakis" to pay tribute to the sparsity principle, and hopes that these models can work more efficiently and reduce costs. In particular, OpenAI's partner Microsoft hopes that "Arrakis" can reduce the cost of using generative AI in its products.
Additionally, Microsoft has shifted resources toward developing cheaper language models, as the high cost of generative AI threatens emerging business models, according to The Wall Street Journal.
Earlier, it was reported that OpenAI leaker FeltSteam had mentioned "Arrakis". The model is said to be able to generate not only text, but also images and videos, with less hallucinations, suitable for autonomous agents. In addition, "Arrakis" will be partially trained with synthetic data and is scheduled to be released next year.