MicrosoftThe company announced the release of three new Phi-3.5 models, further consolidating its leading position in the development of multilingual and multimodal artificial intelligence. The three new models are: Phi-3.5-mini-instruct, -3.5-MoE-instruct and Phi-3.5-vision-instruct, each targeting different application scenarios.
The Phi-3.5Mini Instruct model is a lightweight AI Models, with 380 million parameters, it is very suitable for environments with limited computing power. It supports a context length of 128k and is specially optimized for instruction execution, suitable for tasks such as code generation, mathematical problem solving, and logical reasoning. Despite its small size, this model has shown remarkable competitiveness in multi-language and multi-turn dialogue tasks, surpassing other similar models.
Entry: https://huggingface.co/microsoft/Phi-3.5-mini-instruct
The Phi-3.5MoE model is a "mixture of experts" model that combines multiple different types of models, each focusing on a specific task. It has 41.9 billion parameters, supports 128k context length, and can show strong performance in a variety of reasoning tasks. This model performs very well in code, mathematics, and multi-language understanding, and even surpasses larger models in some benchmarks, such as surpassing OpenAI's GPT-4o mini on MMLU (Massive Multi-Task Language Understanding).
Entry: https://huggingface.co/microsoft/Phi-3.5-MoE-instruct
The Phi-3.5 Vision Instruct model is an advanced multimodal AI model that integrates text and image processing capabilities, suitable for tasks such as image understanding, optical character recognition, chart and table analysis, and video summarization. This model also supports 128k context length and can handle complex multi-frame visual tasks.
Entry: https://huggingface.co/microsoft/Phi-3.5-vision-instruct
To train these three models, Microsoft performed large-scale data processing. The Mini Instruct model used 3.4 trillion tokens and was trained on 512 H100-80G GPUs for 10 days; the Vision Instruct model used 500 billion tokens and was trained for 6 days; and the MoE model used 4.9 trillion tokens for training in 23 days.
It is worth mentioning that all three Phi-3.5 models are released under the MIT open source license, allowing developers to freely use, modify and distribute the software. This not only reflects Microsoft's support for the open source community, but also allows more developers to integrate cutting-edge AI capabilities into their own applications.