iCreatAI News
Skip to content
MIT News AI· Lauren Hinkel | MIT-IBM Computing Research Lab·· 1 hr agoAI score28

Q&A: Enabling more human-like interactions and understanding in multimodal foundation models

Q&A: Enabling more human-like interactions and understanding in multimodal foundation models

AI summary

An MIT-IBM Computing Research Lab team co-led by Jim Glass investigates critical failures of state-of-the-art multimodal models and develops benchmarks and data-efficient methods to address them with limited supervision. The work includes IPLoc, which uses pseudonym substitution to force models to attend to visual context, and OmniR1, which found audio-language models can answer multiple-choice benchmarks without listening to the audio. The team plans to shift focus toward full-duplex spoken dialogue systems while continuing joint audio-visual work.

Source: MIT News AI · news.mit.edu