Training Data

Recently, the shortage of training data for large AI models has once again become the focus of media attention. The Economist magazine's latest article "AI companies will soon exhaust most of the Internet's data" has sparked widespread discussion in the industry. The article points out that as high-quality Internet data runs out, the AI field is facing the challenge of a "data wall." Research company Epoch AI predicts that all high-quality text data on the Internet will be exhausted by 2028, and machine learning data sets may exhaust all "high-quality language data" by 2026. This &qu…

OpenAI recently launched the hot text-to-video generation model Sora, but the company's Chief Technology Officer (CTO) Mira Murati was vague in an interview with the Wall Street Journal and could not clearly explain the source of Sora's training data. During the interview, when the reporter directly asked Murati about the source of Sora's training data, she only used vague official language to prevaricate: "We use publicly available data and licensed data." When the reporter asked whether the specific source included YouTube videos, Murati...

MIT Technology Review once published an article on its official website stating that with the continued popularity of large models such as ChatGPT, the demand for training data is increasing. Large models are like a "network black hole" that constantly absorbs, and will eventually lead to insufficient data for training. The well-known AI research institute Epochai published a paper directly on the data training problem, pointing out that by 2026, large models will consume all high-quality data; by 2030-2050, they will consume all low-quality data; by 2030-2060, they will consume all image training data...