Huazhong University of Science and Technology

Recently, Huazhong University of Science and Technology and other institutions released a new comprehensive evaluation benchmark for large multimodal models (LMMs), aiming to solve the problem of performance evaluation of large multimodal models. This study involved 14 mainstream large multimodal models, including Google Gemini, OpenAI GPT-4V, etc., covering five major tasks and 27 data sets. However, due to the open nature of the answers of large multimodal models, evaluating the performance of various aspects has become an urgent problem to be solved. In this study, special emphasis was placed on the ability of large multimodal models in optical character recognition (OCR). The research team…

Monkey is a high-performance multimodal large model jointly launched by Huazhong University of Science and Technology and Kingsoft Software. By improving the input resolution and introducing a multi-level description generation method, it solves the challenges of existing models in processing complex scenes and visual details. Monkey can be built based on existing visual editors without pre-training from scratch, which greatly improves R&D efficiency. Monkey's multi-level description generation method can provide the model with rich contextual information and guide the model to learn the association between scenes and objects. Through testing on 16 different datasets, Monkey has achieved excellent results in image…