Skip to content

Newly developed optical microphone sees sound like never before

Newly developed optical microphone sees sound like never before

tech innovation 2022

Mark Sheenin (left) and Dorian Chan were part of a CMU research team that developed a camera system that can see sound vibrations with such precision that it can capture different audio from different guitar players playing at the same time. can capture. credit: Carnegie Mellon University

A camera system developed by Carnegie Mellon University researchers can observe sound vibrations with such precision and detail that it can recreate the music of a single instrument in a band or orchestra.

Even the most high powered and guided microphones cannot eliminate the effects of surrounding sounds, ambient noise and acoustics when they capture audio. The novel system developed at the Robotics Institute’s (RI) School of Computer Science uses two cameras and a laser to sense high-speed, low-amplitude surface vibrations. These vibrations can be used to reconstruct sound, capturing isolated audio without any projections or microphones.

“We’ve invented a new way of seeing sound,” said Mark Sheenin, a post-doctoral research associate at the Illumination and Imaging Laboratory (ILIM) in RI. “It’s a new type of camera system, a new imaging device, capable of seeing something invisible to the naked eye.”

The team completed several successful demos of their systems’ effectiveness in sensing vibrations and sound reconstruction quality. They captured different guitars playing different audio at the same time and different speakers playing different music simultaneously. They analyzed the vibrations of a tuning fork, and used the vibrations of a bag of Doritos near a speaker to capture the sound coming from the speaker. This demo pays homage to prior work by MIT researchers who developed one of the first visual microphones in 2014.

The CMU system dramatically improves on previous attempts to capture sound using computer vision. The team’s work uses ordinary cameras that cost a fraction of the high-speed versions employed in previous research while producing high-quality recordings. The dual-camera system can capture vibrations from objects in motion, such as the motion of a guitar while a musician plays it, and simultaneously sense individual sounds from multiple points of view.

“We have made optical microphones more practical and usable,” said Srinivas Narasimhan, professor at RI and head of ILIM. “We’ve made quality better while keeping costs down.”

The system works by analyzing differences in speckle patterns from images captured with the rolling shutter and the global shutter. An algorithm calculates the difference in speckle patterns from two video streams and converts those differences into vibrations to reconstruct the sound.

A speckle pattern refers to the way coherent light behaves in space after being reflected from a rough surface. The team creates the speckled pattern by aiming a laser at the surface of a vibrating object, like the body of a guitar. That patchy pattern changes as the surface vibrates. A rolling shutter captures an image by rapidly scanning it, usually from top to bottom, to produce the image by stacking one row of pixels on top of another. A global shutter captures one image at a time at a time.

The research involved Dorian Chan, Ph.D., along with Sheenin and Narasimhan. student in computer science, and Matthew O’Toole, assistant professor in the Department of RI and Computer Science.

“This system pushes the limits of what can be done with computer vision,” O’Toole said. “This is a new mechanism for capturing high speeds and small vibrations, and presents a new area of ​​research.”

Much of the work in computer vision focuses on training systems to recognize objects or track them through space – important research to advance technologies such as autonomous vehicles. This function enables the system to see imperceptible, high-frequency vibrations opening up new applications for computer vision.

The team’s dual-shutter, optical vibration-sensing system can allow sound engineers to monitor the music of individual instruments, free of interference from the rest of the ensemble, to fine-tune the overall mix. Manufacturers can use systems to monitor the vibrations of individual machines on the factory floor to look for early signs of necessary maintenance.

“If your car starts making a strange sound, you know it’s time to look at it,” Sheinin said. “Now imagine a factory floor full of machines. Our system allows you to monitor the health of each one by sensing their vibration with a stationary camera.”


drawing a figure based on sound


more information:
Mark Sheenin et al, Dual-shutter optical vibration sensing(2022)

Provided by Carnegie Mellon University

Citation: Newly developed optical microphone sees sound like never before (2022, 21 June) Retrieved 21 June 2022

This document is subject to copyright. No part may be reproduced without written permission, except for any fair use for the purpose of personal study or research. The content is provided for information purposes only.

credit source

Newly developed optical microphone sees sound like never before

#Newly #developed #optical #microphone #sees #sound

if you want to read this article from the original credit source of the article then you can read from here

Shopping Store 70% Discount Offer

Leave a Reply

Your email address will not be published.