In training, the network is provided such pairs of audio until it learns to produce embeddings with this property. Pairs of audio containing different melodies should be far apart. To enable humming recognition, the network should produce embeddings for which pairs of audio containing the same melody are close to each other, even if they have different instrumental accompaniment and singing voices. A neural network is trained with pairs of input (here pairs of hummed or sung audio with recorded audio) to produce embeddings for each input, which will later be used for matching to a hummed melody. In principle, many such retrieval systems (e.g., image recognition) work in a similar way. The first step in developing Hum to Search was to modify the music-recognition models used in Now Playing and Sound Search to work with hummed recordings. The next challenge then was to leverage what was learned from these releases to recognize hummed or sung music from a similarly large library of songs. Now Playing, released to Pixel phones in 2017, uses an on-device deep neural network to recognize songs without the need for a server connection, and Sound Search further developed this technology to provide a server-based recognition service for faster and more accurate searching of over 100 million songs. Prior efforts to enable discovery of music, in particular in the context of recognizing recorded music being played in an environment such as a cafe or a club, demonstrated how machine learning might be applied to this problem. To find by eye the dominant melody that might be used to match these two spectrograms, a person might look for similarities in the lines near the bottom of the above images.
To achieve this, the model has to learn to focus on the dominant melody, and ignore background vocals, instruments, and voice timbre, as well as differences stemming from background noise or room reverberations. Given the image on the left, a model needs to locate the audio corresponding to the right-hand image from a collection of over 50M similar-looking images (corresponding to segments of studio recordings of other songs). Visualization of a hummed clip and a matching studio recording. The difference between the hummed version and the same segment from the corresponding studio recording can be visualized using spectrograms, seen below: However, one challenge in recognizing a hummed melody is that a hummed tune often contains relatively little information, as illustrated by this hummed example of Bella Ciao. Many existing music recognition systems convert an audio sample into a spectrogram before processing it, in order to find a good match. This approach greatly simplifies the database for Hum to Search, allowing it to constantly be refreshed with embeddings of original recordings from across the world - even the latest releases. This enables the model to match a hummed melody directly to the original (polyphonic) recordings without the need for a hummed or MIDI version of each track or for other complex hand-engineered logic to extract the melody. In contrast to existing methods, this approach produces an embedding of a melody from a spectrogram of a song without generating an intermediate representation. Launched in October, Hum to Search is a new fully machine-learned system within Google Search that allows a person to find a song using only a hummed rendition of it.
GOOGLE MUMBLE SONG MANUAL
However, this type of approach often relies on a limited database that requires manual updates. That’s why so many existing approaches to query by humming match the hummed tune against a database of pre-existing melody-only or hummed versions of a song, instead of identifying the song directly. By mistake or design, when someone hums their interpretation of a song, often the pitch, key, tempo or rhythm may vary slightly or even significantly. With lyrics, background vocals and instruments, the audio of a musical or studio recording can be quite different from a hummed tune. But what if you can’t quite recall the name of the song, and can only hum the melody?Įxisting methods to match a hummed melody to its original polyphonic studio recording face several challenges. Research has found that engaging with the original song, whether that’s listening to or singing it, will drive the earworm away. Melodies stuck in your head, often referred to as “ earworms,” are a well-known and sometimes irritating phenomenon - once that earworm is there, it can be tough to get rid of it. Posted by Christian Frank, Google Research, Zürich
0 kommentar(er)
