According to TechCrunch.MuskAgree with other AI experts that real-world use for training AI ModelsofdataIt's almost depleted.
In a live conversation with Mark Payne, chairman of Stagwell's board of directors, on Wednesday evening, Musk said, "We've now essentially consumed all of the accumulated human knowledge ...... of data used for AI training. This phenomenon basically happened last year."
Musk's comments are similar to those made by former OpenAI chief scientist Ilya Sutskever at the NeurIPS conference last December. Sutskever noted that the AI industry has beenThe so-called "data peak" was reached., and predicts that the lack of sufficient training data in the future will force changes in the way AI models are developed.
Musk believes that synthetic data (note: i.e., data that is self-generated by AI models) is the solution of the future. "The only way to supplement real-world data is through synthetic data, which means making AI Generate your own training data. the AI will self-assess and continue to optimize itself through this self-learning process."
Many tech companies, including Microsoft, Meta, OpenAI, and Anthropic, are now using synthetic data to train their workhorse AI models. Gartner estimates that by 2024, the AI and data analytics programs used for 60% data will be generated synthetically.
One significant advantage of using synthetic data is cost reduction. Artificial intelligence startup Writer says its Palmyra X 004 model relies almost entirely on synthetic data for development, with development costsOnly $700,000The development cost of a similarly sized OpenAI model is approximately $4.6 million.
However, there are some risks associated with synthetic data. Studies have shown that synthetic dataMay cause model performance degradationOutput resultsNot only does it lack innovation, but it may become even more biasedthat ultimately seriously affects its functionality. Because the model is trained by generating synthetic data itself, if that data is inherently biased or limited, the output of the final model will be affected by those factors as well.