Small Language Model

Zyphra recently officially launched Zamba2-7B, a small language model with unprecedented performance and a parameter count of 7B. The model is claimed to outperform current competitors, including Mistral-7B, Google's Gemma-7B, and Meta's Llama3-8B, in terms of quality and speed. Zamba2-7B is designed The goal is to satisfy environments that require robust language processing power but are hardware-constrained, such as on-device processing or the use of consumer-grade GPUs.By increasing efficiency without sacrificing...

The recent release of the new Zamba2-2.7B language model from Zyphra is significant in the history of small language modeling. The new model achieves significant improvements in performance and efficiency with a training dataset of approximately 3 trillion tokens, making it comparable in performance to Zamba1-7B and other leading 7B models. And most surprisingly, Zamba2-2.7B has significantly lower resource requirements for inference, making it an efficient solution for mobile device applications. Zamba2-2.7B ...

Microsoft Research announced Tuesday local time its Phi-2 Small Language Model (SML), a text-to-text AI program that is "small enough to run on a laptop or mobile device," according to a post on the X platform. With 2.7 billion parameters, Phi-2 rivals the performance of other, larger models, including Meta's Llama 2-7B (with 7 billion parameters) and Mistral-7B (another 7 billion parameter model). Microsoft Research also noted in its blog post on the Phi-2 release that...