OpenBuddy The team released the Meta Llama370B-basedup to dateResults:OpenBuddy-Llama3-70B-v21.1-8k, this is theirfirstChinese cross-language training attempts on the Llama 370B base.
Llama370B is a 15T training set trained on a denseLarge ModelHowever, due to the lack of Chinese data in the training set, Meta's official Llama3 series of dialog models have weak cognitive and reasoning abilities in the Chinese domain and are prone to answering Chinese questions in English.
Therefore, the OpenBuddy team chose the Llama370B Base model as the basis for this training, focusing on improving the model's cross-language comprehension and cognitive abilities.
During the test, OpenBuddy-Llama3-70B demonstrated excellent Chinese comprehension with more cognitive potential and post-quantization performance. In this release, they expanded the existing word list of Llama3 by adding more than 7,000 Chinese words to further improve the Chinese coding density of the model.
It is worth mentioning that, in the quantized version of the model capacity decline is more obvious, they only released the full copyright weight of the model, and will taste the optimization of the 70B model quantized performance, and pick the right time to release the quantized version.