MetaThe company announced a major technological advancement with the release of itsup to dateThe open source model of Llama3 is released. This model contains two parameter sizes: 8B (8 billion parameters) and 70B (70 billion parameters), and is expected to be officially released in July this year. The Llama3 model not only has multimodal capabilities, but also integrates new computer coding functions and can output text and images, indicating that it will be used in devices such as Ray-Ban Meta smart glasses in the future.
Meta has established a partnership with Google to integrate real-time search results into the AI assistant's replies, further improving the user experience. Compared with the previous generation Llama2, Llama3 has demonstrated breakthrough performance in multiple industry benchmarks, especially in reasoning, code generation, and instruction tracing. In order to train this model, Meta built two clusters containing 24,000 GPUs, of which 16,000 GPUs were used for Llama3 training at the same time.
The development of Llama3 foreshadows Meta’s ambitions in the field of AI.maximumThe model has a parameter size of over 400B (40 billion). The main performance features of Meta Llama3 include:
Model size and type: Provides pre-trained and instruction-tuned generative text models in two sizes: 8B and 70B, optimized for conversations and other language processing tasks.
Model architecture: We adopt an autoregressive approach and an optimization-based transformer architecture, introduce a grouped query attention mechanism, and improve the scalability of reasoning under large-scale parameters.
Training and data processing: Use more than 15 trillion tokens for pre-training, the training set is seven times that of Llama2, and supports up to 30 languages.
Performance and Benchmarks: It performs well on multiple automated benchmarks, including dialogue, question-answering, reasoning, and other tasks, and is specially optimized for real-world scenarios.
Meta also released an AI assistant based on Llama3, which is now available in more than a dozen countries outside the United States. Users can use the Meta AI assistant on Facebook, Instagram, WhatsApp and Messenger, which can recommend restaurants, plan weekend activities, assist with learning, etc. In addition, Meta AI also integrates search functions, which can directly access real-time information on the network without switching applications.
Experience address: https://www.meta.ai/
Model download: https://llama.meta.com/llama3/