Matra Studio listens to a recording and writes down what it hears. Most of that listening is done by models that other people built and chose to share. This page names them, says what each one does for you, and points you to the people behind it.
Each license below is the one the project’s authors publish. Where a project shares its code and its trained model on different terms, or does not spell the terms out, we say so and link to the source.
Chords
- ChordMini, by Nghia Phan and the ChordMini contributors. It listens for the harmony in your recording and names the chords. Nghia Phan has told us we are welcome to use his trained chord model in Matra Studio, commercial use included, and that a credit is appreciated. Thank you. The code is released under the MIT license.
Beats and bars
- Beat This, by Francesco Foscarin, Jan Schlüter and Gerhard Widmer, released by the Institute of Computational Perception at Johannes Kepler University Linz. It finds the beats and where each bar begins, which is how Matra draws bar lines and keeps a chart in time. Its code and its trained model are both released under the MIT license.
Separated parts
When you solo the vocals or mute the drums, you are hearing the work of these projects.
- SCNet, by Weinan Tong and co-authors. It pulls a recording apart into vocals, drums, bass and everything else. The code is released under the MIT license.
- Music-Source-Separation-Training, by ZFTurbo. He trained and shared the SCNet XL IHF model we use, and asked to be credited by that name. His toolkit is released under the MIT license, and the trained model is used under the terms he publishes.
- Demucs, by Alexandre Défossez, Simon Rouard and Francisco Massa, released through Meta AI Research. It was the first separator in our engine and still ships alongside SCNet. The code is released under the MIT license, and the trained model is used under the terms its authors publish.
Notes
- YourMT3+, by Sungkyun Chang, Emmanouil Benetos, Holger Kirchhoff and Simon Dixon. It listens to the whole mix and writes down which notes each instrument plays. The code is on GitHub under GPL-3.0, and the trained checkpoints are shared on Hugging Face with an Apache-2.0 license tag.
- Basic Pitch, from Spotify’s Audio Intelligence Lab. A light and quick note detector that helps follow the melody. Released under the Apache 2.0 license.
Singing voice
- RMVPE, by Haojie Wei, Xueke Cao, Tangpeng Dan and Yueguo Chen. It follows the pitch of a singing voice even with instruments playing underneath, which helps Matra place the notes of a sung melody. The reference code is released under the Apache 2.0 license.
- Retrieval-based Voice Conversion (the RVC Project). Our voice-pitch code is adapted from theirs, under the MIT license. The trained RMVPE model is hosted in the Hugging Face repository that project uses, and that page declares the MIT license.
Everything underneath
- PyTorch runs the models above. It has a BSD-style license.
- librosa reads and prepares the audio. ISC license.
- Verovio, from the RISM Digital Center, engraves the staff notation you see in Studio. Available under the LGPL.
Something missing?
If we have left someone out or got a detail wrong, tell us and we will fix it. The datasets and fonts we use are listed under Licenses & attributions.