recently,HKUand a team of researchers at NYU have collaborated to develop a platform called V-IRL, which incorporates real-world maps, street views, and other rich information into virtual environments for theAI AgentProvides a more authentic and complete life experience.
This research centers on building a scalable platform to embed AI intelligences into real urban environments using APIs such as geographic information and street view images. The researchers used V-IRL to instantiate a series of intelligences that can perform a variety of tasks, such as finding restaurants, housing, and urban planning. On this platform, the intelligences can also collaborate with each other to solve complex problems.
Project address:https://github.com/VIRL-Platform/VIRL
In addition, the research team conducted V-IRL-based benchmark tests for place recognition, visual quizzing, and navigation. The results showed that the visual models faced challenges in processing multilingual image data, especially in some geographically diverse cities, where the models did not perform as well as they could have.
However, these challenges have not stopped the V-IRL platform from offering vast possibilities for AI applications in the real world. By evaluating the performance of different tasks, the researchers demonstrated the platform's potential for a wide range of applications of language and visual modeling in global visual data processing, opening up new possibilities for improving AI's ability to understand the environment, make decisions, and process real-world information. With the increasing popularity of spatial computing technology and robotic systems, the demand and application scenarios for AI Agents will continue to expand, and we look forward to the era of an intelligent body that can deeply understand the world around it.
The significance of this research is that it frees AI intelligences from traditional virtual environments, allowing them to learn and interact in environments similar to the real world. By combining with the real world, AI Agents can better understand human behaviors and needs, thus providing more accurate and efficient solutions for a variety of practical tasks. At the same time, this research also demonstrates the great potential of AI in the future in the fields of urban planning, personal assistants, life aids, etc., laying a solid foundation for us to enter the era of intelligent bodies.