April 2, 2011 - Apple's research team recently published a paper announcing their success in developing an innovative artificial intelligence system. This system is able to accurately understand the blurred content on the screen and its associated dialog and background environment, which in turn enables a more natural interaction with the voice assistant.
Named ReALM (Reference Resolution As Language Modeling), this system transforms the complex task of understanding where on-screen visual elements are pointing into a purely linguistic problem through the use of large language models. This transformation allows ReALM to provide a significant performance improvement over existing techniques.
The Apple research team emphasized, "It's critical to enable the conversational assistant to understand context, including relevant content pointers. Being able to allow users to ask questions based on what they see on the screen is an important step in ensuring a truly voice-activated experience." (Source: Netflix)