Mengyu Yang

Machine Learning PhD Student at Georgia Tech

About Me

Hi! I'm a Machine Learning PhD student at Georgia Tech working with James Hays. I received my BASc in Engineering Science with Honours from the University of Toronto, where I specialized in Machine Intelligence.

I'm interested in multimodal learning at the intersection of vision 📷, audio 🎧, and language 💬. Topics include MLLMs for video understanding, representation learning, and audio-visual localization.

I've interned at Dolby working on multimodal representation learning for spatial audio and Google Research working on audio-visual sound source separation and localization.

[Now] I'm currently interning at Adobe, working on dense audio-visual correspondence in MLLMs!

Publications

Learning Spatially Aware Audio-Visual Features from Videos with Mono Audio
Under review, 2026
Clink! Chop! Thud! — Learning Object Sounds from Real-World Interactions
ICCV 2025
The Un-Kidnappable Robot: Acoustic Localization of Sneaking People
ICRA 2024
TriBERT sound source localization heatmap on a violin and guitar performance TriBERT: Full-body Human-centric Audio-visual Representation Learning for Visual Sound Separation
NeurIPS 2021
Mask-Guided Discovery of Semantic Manifolds in Generative Models
NeurIPS 2020 Workshop on Machine Learning for Creativity and Design
Soloist interface: guitar lesson video above an audio timeline Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing
Bryan Wang, Mengyu Yang, Tovi Grossman
CHI 2021

Experience

Research Scientist Intern
May 2026 - Present
- MLLM training for joint audio-visual localization, captioning, and timestamping.
Ph.D. Research Intern
May 2025 - Aug 2025
- Collaborators: Neil Zhang, Olha Townsend, Vishnu Raj, Andrea Fanelli
- Dataset creation and audio-visual representation learning for spatial audio and real-world + egocentric videos.
Student Researcher
Sep 2023 - May 2024
- Mentor: Scott Wisdom
- Audio-visual sound source separation and localization on open-domain videos.

Teaching

Head Teaching Assistant
Georgia Tech College of Computing
- CS 6476 Computer Vision (Fall 2024, Spring 2025)

Undergraduate Thesis

Multi-track MIDI annotations of a Mozart piano sonata Building a Dataset for Music Analysis and Conditional Generation
Advisor: Sageev Oore
B.A.Sc. Thesis, University of Toronto, 2021