To help the visually impaired "see" the world, the Fudan University team developed the "Mousi" large model and the "Hear the World" App

according toFudan UniversityThe official public account is made possible by the efforts of teachers and students of Fudan University Natural Language Processing Laboratory (FudanNLP).Based on multimodalLarge Model"Fudan MouSi" launches the "Hear the World" app tailored for the visually impaired.

To help the visually impaired "see" the world, the Fudan University team developed the "Mousi" large model and the "Hear the World" App

This system only needs a camera and a pair of headphones to convert images into language, and supports functions such as describing scenes and warning of risks. The "Hear the World" App can design three modes for the daily life needs of the visually impaired.

  • Street Walking: In this mode,"Mosi" can scan road conditions in detail and indicate potential risks.

  • Free Q&A: It can help the visually impaired walk into museums, art galleries, and parks, capture every detail of the surrounding scenes, and use sound to build rich life scenes. The official demonstration picture shows thatThe app can also realize functions such as retelling TV screen content.

  • Object Search: This mode provides the visually impaired with the function of finding everyday objects, and the official calls it a “reliable butler.”

It is reported that the "Hear the World" App is expected to complete its first round of testing in March this year, and will simultaneously launch pilot projects in China's first- and second-tier cities and regions, and promote it based on the computing power deployment situation.

Fudan University Natural Language Processing Laboratory (FudanNLP) previously developed the MOSS large model and announced its official open source release in April 2023.Became the first plug-in enhanced open source conversational language model in China. Half a year later, the multimodal model "Mosi" was launched.

statement:The content is collected from various media platforms such as public websites. If the included content infringes on your rights, please contact us by email and we will deal with it as soon as possible.
HeadlinesInformation

Violating citizens' personal information and illegally earning 35,000 yuan, 3 people were sentenced for using face-changing software to help others unblock their accounts

2024-3-3 9:10:55

Information

"4K HD version" meteorological model: Shanghai Artificial Intelligence Laboratory "Fengwu" achieves 10 km level weather forecast

2024-3-3 9:12:41

Search