Open Source Big Model

XVERSE released XVERSE-MoE-A36B, the largest MoE open source model in China. The model has 255B total parameters and 36B activation parameters, and it is officially claimed that the effect can "roughly reach" more than the "cross-level" performance leap of the 100B large model, while the training time is reduced by 30%, and the inference performance is improved by 100%. "At the same time, the training time is reduced by 30%, the inference performance is improved by 100%, and the cost per token is greatly reduced. MoE (Mixture of Experts) is a hybrid expert modeling architecture that combines multiple segmented expert models into a single super model, expanding the model size...

In the field of AI, a new force is rising. Abu Dhabi Technology Innovation Institute (TII) announced the open source of their new large model Falcon2, a model with 11 billion parameters, which has attracted global attention with its excellent performance and multilingual capabilities. Falcon2 has two versions, a basic version that is easy to deploy and can generate text, code and summaries; the other is a VLM model with visual conversion function that can convert image information into text, which is extremely rare in open source large models. In multiple rights protection test rankings, the performance of Falcon211B surpassed...

SemiKong, the world's first open source chip design model, has been officially released. It is fine-tuned based on Llama3 and its performance exceeds that of general large models. This is not only a technological breakthrough, but also a victory for the open source spirit, which indicates that the $500 billion semiconductor industry will usher in earth-shaking changes in the next five years. The advent of SemiKong marks a solid step forward in the application of AI in the field of chip design. It was jointly created by Aitomatic and FPT Software and made its debut at the Semicon West2024 conference, which attracted the attention of the industry...

Tencent has released an acceleration library for the Tencent Hunyuan Text-to-Image Open Source Model (Hunyuan DiT), claiming to significantly improve inference efficiency and shorten image generation time by 75%. Officials said that the threshold for using the Hunyuan DiT model has also been greatly reduced, and users can use the Tencent Hunyuan Text-to-Image model capabilities based on the ComfyUI graphical interface. At the same time, the Hunyuan DiT model has been deployed to the HuggingFaceDiffusers general model library. Users can call the Hunyuan DiT model with only three lines of code without downloading the original code library. Prior to this, Tencent…

Mianbi Intelligence released the open source large model Eurux-8x22B, including Eurux-8x22B-NCA and Eurux-8x22B-KTO, focusing on reasoning ability. In the official test, Eurux-8x22B surpassed Llama3-70B in LeetCode (180 LeetCode programming questions) and TheoremQA (IT Home Note: STEM questions at the level of American universities) tests, and surpassed the closed source GPT-3.5-T in LeetCode tests...

On April 19, Baidu Smart Cloud Qianfan Big Model Platform launched the first training and reasoning solution for the full series of Llama3 versions in China, which is convenient for developers to retrain and build exclusive big models. It is now open for invitation testing. At present, ModelBuilder, a custom tool for models of various sizes in Baidu Smart Cloud Qianfan Big Model Platform, has pre-set the most comprehensive and rich big models, supporting mainstream third-party models at home and abroad, with a total of 79 models, making it the development platform with the largest number of big models in China. It is reported that on April 18, Meta officially released Llama3, including 8B …

At present, it has become an important trend to integrate large language models into the programming field and complete code generation and completion tasks. A number of eye-catching large code models have emerged in the industry, such as OpenAI's CodeX, Google DeepMind's AlphaCode, and HuggingFace's StarCoder, which help programmers complete coding tasks more quickly, more accurately, and with higher quality, greatly improving efficiency. There is such a research and development team that began to explore the use of deep learning in software development 10 years ago, and in the fields of code understanding and code generation...

Meta Platforms Inc. plans to launch two small-parameter versions of the Llama3 Large Language Model (LLM) next week as a precursor to the upcoming maximum version of Llama3, which is due to be launched in the summer of 2024, according to foreign media reports. The top version of Llama3 could reportedly have more than 140 billion parameters, which would put its performance on track to catch up with OpenAI's latest GPT-4Turbo version. However, Meta's version to be announced next week does not support multimodal technology at this time. This news could spark strong anticipation for Llama3. Last July, Ll...

Big data company Databricks has sparked a buzz in the open source community with the recent release of a MoE big model called DBRX, which has beaten open source models such as Grok-1 and Mixtral in benchmarks to become the new open source king. The model has 132 billion total parameters, but only 36 billion parameters per activation, and it generates them 1x faster than Llama2-70B. DBRX is composed of 16 expert models with 4 experts active per inference and a context length of 32 K. To train DBRX, Data...

The official WeChat account of "Zero One Thousand Things 01AI" announced the open source Yi-9B model tonight. The official called it the "Science Champion" in the Yi series of models. Yi-9B is currently the model with the strongest code and mathematical capabilities in the Yi series of models, with an actual parameter of 8.8B and a default context length of 4K tokens. This model is based on Yi-6B (trained with 3.1T tokens) and uses 0.8T tokens for further training. The data is as of June 2023. According to the introduction,…

On November 10, at the 2023 World Internet Conference Wuzhen Summit Internet Entrepreneur Forum held yesterday, Alibaba Group CEO Wu Yongming delivered a speech. Wu Yongming said that Alibaba is about to open source a large model with 72 billion parameters, which will be the largest open source large model with the largest number of parameters in China. This is not the first time that Alibaba has open sourced a large model. In August of this year, Alibaba launched two open source models, Qwen-7B and Qwen-7B-Chat, which are the Tongyi Qianwen 7 billion parameter general model and dialogue model. In September of this year, Alibaba also open sourced the Tongyi Qianwen 14...