4-5:15pm EST Monday, Nov. 24 in ITE229 and online

Whether tracking emerging events, analyzing economic trends, or understanding public discourse, valuable information is scattered across modalities, from professionally produced news content and curated Wikipedia articles to firsthand footage of disasters livestreamed on social media. Building systems that can effectively retrieve, reason over, and synthesize these heterogeneous information sources is essential for knowledge-intensive applications.

This talk will focus on advancing both sides of the information-seeking pipeline: retrieving relevant multimodal evidence at scale, and synthesizing that evidence into coherent, Wikipedia-style explanations grounded in verifiable evidence. For retrieval, we will focus on recent progress in large-scale multimodal retrieval, including new dataset, efficient and scalable first-stage retrieves, and reasoning reranking. In Wikipedia-style article generation, we will cover benchmarking and evaluating multimodal article generation and a method for enabling the use of VLMs for high-level reasoning. Together, these components outline a path toward unified systems capable of transforming large collections of multimodal evidence into verifiable, human-readable articles.

Alexander Martin is a PhD candidate at Johns Hopkins University’s Center for Language and Speech Processing (CLSP) and Human Language Technology Center of Excellence (HLTCOE). He is advised by Dr. Benjamin Van Durme. Alex’s research focuses on end-to-end multimodal information retrieval and reasoning. His work aims to produce Wikipedia-style articles, grounded in retrieved documents and videos, in response to information seeking queries. His research has been published in CVPR, ACL, NAACL, and EMNLP. Alex is a recipient of the NSF’s Graduate Research Fellowship.

Hosted by Prof. Tejas Gokhale at UMBC ITE 229 and online.