iCreatAI News
Skip to content
Trending storyUpdating

MIT-IBM studies multimodal model understanding deficiencies

1 reports1 sources52 min agoUpdated

Understand this story

AI overview

A team at the MIT-IBM Computing Research Lab, led by Jim Glass, studies deficiencies in genuine understanding in multimodal foundation models and develops new benchmarks and low-annotation-data methods to improve them. The team says this work targets the goal of making model interaction and understanding "more human-like"; the report does not give specific performance data for the new benchmarks or methods.

AI overview based on source reports · Updated 17 min ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 21
  1. MIT News AI
    Q&A: Enabling more human-like interactions and understanding in multimodal foundation models

    An MIT-IBM Computing Research Lab team co-led by Jim Glass investigates critical failures of state-of-the-art multimodal models and develops benchmarks and data-efficient methods to address them with limited supervision. The work includes IPLoc, which uses pseudonym substitution to force models to attend to visual context, and OmniR1, which found audio-language models can answer multiple-choice benchmarks without listening to the audio. The team plans to shift focus toward full-duplex spoken dialogue systems while continuing joint audio-visual work.

Activity for this story

Not enough continuous observations to show a trend yet.