Multimodal Model

Google today launched its latest AI masterpiece, Gemini 1.5 Pro, and made an experimental “version 0801” available through Google AI Studio and the Gemini API for early testing and feedback. The new model quickly topped the famous LMSYS Chatbot Arena leaderboard (built by Gradio) with an impressive ELO score of 1300. This achievement puts Gemini 1.5 Pro ahead of OpenAI’s GPT…

Meta recently released a multimodal model called Chameleon, which sets a new benchmark in the development of multimodal models. Chameleon is a family of early-fusion token-based mixed-modal models that can understand and generate images and text in any order. It uses a unified Transformer architecture to complete training using mixed modalities of text, images, and code, and tokenizes images to generate interleaved text and image sequences. The innovation of the Chameleon model lies in its early fusion method, where all processing pipelines are synchronized from the beginning...

The latest flagship end-side model launched by Mianbi Intelligent - Mianbi MiniCPM2.0 series models bring a series of amazing performance and functions: 1. MiniCPM-V2.0 is the strongest multimodal model on the end-side, with powerful OCR capabilities, and even some capabilities are comparable to Gemini Pro. It can accurately recognize various complex image contents, including street scenes and long pictures, through its self-developed high-definition image decoding technology. 2. MiniCPM-1.2B is a base model that is more suitable for end-side scenarios, and its performance exceeds many mainstream models,…

In the field of artificial intelligence, the development of multimodal models has always been the focus of industry attention. Recently, Musk XAI released its latest multimodal model, Grok-1.5Vision, which can not only process text information, but also understand and analyze various visual data, such as documents, charts, screenshots and photos, marking an important step for the company in artificial intelligence technology. The Grok-1.5Vision model has demonstrated its excellent performance in multiple benchmark tests. Compared with the industry-leading GPT4V model, it is not only comparable, but even achieves multiple indicators...

After Musk's artificial intelligence company xAI launched the Grok-1.5 large language model in late March, it recently launched its first multimodal model, Grok-1.5 Vision. xAI said it will invite early testers and existing Grok users to test Grok-1.5 Vision (Grok-1.5V) in the near future. It can not only understand text, but also process the content in documents, charts, screenshots and photos. xAI said: "Grok-1.5V has excellent performance in multidisciplinary reasoning, document understanding, scientific charts, table processing...

Recently, Zero One Everything API was officially opened to developers, which includes three powerful models. The first is Yi-34B-Chat-0205, which supports general chat, Q&A, dialogue, writing and translation functions; the second is Yi-34B-Chat-200K, which can handle multi-document reading comprehension and build a super-long knowledge base; the last is Yi-VL-Plus multimodal model, which supports text and visual multimodal input, and the Chinese chart experience exceeds GPT-4V. The opening of these models will promote the implementation of a wider range of application scenarios and form a more prosperous ecosystem. Address: https:…

Recently, Shengshu Technology's "Multimodal Big Model" was officially registered with the national "Interim Measures for the Management of Generative Artificial Intelligence Services". Founded in March 2023, Shengshu Technology is a world-leading artificial intelligence company that independently develops multimodal general big models. It has deployed MaaS (model as a service) and application-level products to empower art design, game production, film and television animation, social entertainment and other fields. In March 2023, the team adopted the native technology route and released and open-sourced the world's first multimodal model UniDiffuser based on the U-ViT architecture, successfully converting the diffusion model for the first time...

Microsoft has open-sourced the multimodal model LLaVA-1.5, which inherits the LLaVA architecture and introduces new features. Researchers have tested it in visual question answering, natural language processing, image generation, etc. and found that LLaVA-1.5 has reached the highest level among open-source models, comparable to the effect of GPT-4V. The model consists of three parts: the visual model, the large language model, and the visual language connector. Among them, the visual model uses the pre-trained CLIP ViT-L/336px. Through CLIP encoding, a fixed-length vector representation can be obtained to improve the representation of image semantic information. Compared with the previous version…